mirror of
https://github.com/euzu/tuliprox.git
synced 2026-09-28 20:12:12 +02:00
Feature/staged input and path improvements (#632)
* No template file not found error message * Setup wizard improvmenets * path improvements * staged input fix * path improvements * Provider dns resolved ips now writing to a file instead of source.yml * Playlist update hard kill after 1h * matadata update tmdb fix * Web UI fixes * GeoIp update schedule * batch:// prefix for aliases
This commit is contained in:
@@ -4,6 +4,7 @@
|
||||
|
||||
## ⚠️ Breaking Changes
|
||||
|
||||
- `working_dir` in `config.yml` renamed to `storage_dir`.
|
||||
- **Global Input Definitions**: To align input definitions with the SourceEditor, inputs are now defined globally in the `inputs` section of the
|
||||
config file. Each source can reference one or more inputs by their name in the `inputs` attribute.
|
||||
- **Data Format Migration**: Due to heavy refactoring, the old data format is invalid. You need to clean your `data` folder and update the playlists.
|
||||
@@ -38,6 +39,27 @@
|
||||
fallback_to_filename: true
|
||||
```
|
||||
|
||||
- **Input Batch URL Scheme**: Batch input URLs now use the `batch://` scheme instead of `file://`.
|
||||
`file://` is no longer accepted for batch CSV definitions. Update your `source.yml`:
|
||||
|
||||
```yaml
|
||||
# Before
|
||||
inputs:
|
||||
- type: xtream_batch
|
||||
url: 'file:///home/tuliprox/config/batch.csv'
|
||||
|
||||
# After
|
||||
inputs:
|
||||
- type: xtream_batch
|
||||
url: 'batch:///home/tuliprox/config/batch.csv'
|
||||
```
|
||||
|
||||
Local paths without a scheme (`/path/file.csv`, `./file.csv`) continue to work.
|
||||
The `batch://` scheme clearly distinguishes batch alias files from provider `file://` URLs.
|
||||
- **DNS Resolved Persistence**: `dns.resolved` has been removed from `source.yml` and the `ProviderDnsDto`.
|
||||
Resolved IPs are now persisted separately in `{storage_dir}/provider_dns_resolved.json`.
|
||||
This eliminates hot-reload interference caused by DNS refresh cycles writing to `source.yml`.
|
||||
DNS caches are automatically carried over during config hot-reloads.
|
||||
- **Input Batch Changes**: `name` attribute is now mandatory for input type batch to ensure stable playlist UUIDs.
|
||||
- **Favorites Redesign**: Replaced implicit `create_alias` with explicit `add_favourite(group_name)` script function.
|
||||
- **EpgSmartMatch**: Field `name_prefix` syntax needs to be changed from `name_prefix: !suffix "."` to `name_prefix: { suffix: "." }`.
|
||||
@@ -104,6 +126,7 @@
|
||||
- **Playlist Caching**: Added `cache_duration` to inputs, allowing configurable provider playlist cache times during subsequent updates (e.g., `60s`,
|
||||
`5m` `12h`, `1d`).
|
||||
- **Database Viewer**: New CLI flags `--dbx` and `--dbm` to inspect internal database content.
|
||||
- **Home Directory Override**: Added `--home` (`-H`) CLI argument to set the base directory for config, data, backup, and downloads.
|
||||
- **Added `disk_based_processing`**: (boolean, default `false`) to `config.yml`. When enabled, input playlists are processed from disk instead of
|
||||
memory.
|
||||
- **User-Agent `default_user_agent`**: Ensures that outgoing requests always pass a default user agent.
|
||||
|
||||
@@ -61,6 +61,7 @@ Define complex filters using expressive logic, e.g.:
|
||||
Usage: tuliprox [OPTIONS]
|
||||
|
||||
Options:
|
||||
-H, --home <HOME> The home directory (base for config, storage, backup, downloads, web)
|
||||
-p, --config-path <CONFIG_PATH> The config directory
|
||||
-c, --config <CONFIG_FILE> The config file
|
||||
-i, --source <SOURCE_FILE> The source config file
|
||||
@@ -83,10 +84,27 @@ Options:
|
||||
|
||||
```
|
||||
|
||||
Home directory resolution order:
|
||||
|
||||
1. `--home`
|
||||
2. `TULIPROX_HOME` environment variable
|
||||
3. Directory of the `tuliprox` binary
|
||||
|
||||
All relative paths that are resolved against `home` (including `storage_dir` when configured as relative) use this order.
|
||||
|
||||
Default layout under `{home}`:
|
||||
|
||||
- `{home}/config`
|
||||
- `{home}/data` (`storage_dir` default)
|
||||
- `{home}/data/backup` (`backup_dir` default = `{storage_dir}/backup`)
|
||||
- `{home}/downloads`
|
||||
- `{home}/web`
|
||||
|
||||
## 1. `config.yml`
|
||||
|
||||
For running in cli mode, you need to define a `config.yml` file which can be inside config directory next to the executable or provided with the
|
||||
`-c` cli argument.
|
||||
For CLI mode, `config.yml` is loaded from `{home}/config/config.yml` by default.
|
||||
`{home}` is resolved in this order: `--home` -> `TULIPROX_HOME` -> directory of the `tuliprox` binary.
|
||||
You can override the config file with `-c` (and config directory with `-p`).
|
||||
|
||||
For running specific targets use the `-t` argument like `tuliprox -t <target_name> -t <other_target_name>`.
|
||||
Target names should be provided in the config. The -t option overrides `enabled` attributes of `input` and `target` elements.
|
||||
@@ -95,7 +113,7 @@ This means, even disabled inputs and targets are processed when the given target
|
||||
Top level entries in the config files are:
|
||||
|
||||
- `api`
|
||||
- `working_dir`
|
||||
- `storage_dir`
|
||||
- `default_user_agent` _optional_, used as fallback for upstream requests when no `User-Agent` is provided by input headers or the client request
|
||||
(client request overrides it).
|
||||
- `process_parallel` _optional_
|
||||
@@ -132,12 +150,16 @@ If you process the same provider multiple times each thread uses a connection. K
|
||||
`api` contains the `server-mode` settings. To run `tuliprox` in `server-mode` you need to start it with the `-s`cli argument.
|
||||
-`api: {host: localhost, port: 8901, web_root: ./web}`
|
||||
|
||||
### 1.3. `working_dir`
|
||||
`web_root` follows the same home-based resolution rules:
|
||||
with `web_root: ./web`, the effective path is `{home}/web` (`--home` -> `TULIPROX_HOME` -> binary directory).
|
||||
|
||||
`working_dir` is the directory where files are written which are given with relative paths.
|
||||
-`working_dir: ./data`
|
||||
### 1.3. `storage_dir`
|
||||
|
||||
With this configuration, you should create a `data` directory where you execute the binary.
|
||||
`storage_dir` is the directory where files are written which are given with relative paths.
|
||||
-`storage_dir: ./data`
|
||||
|
||||
`storage_dir` is resolved relative to `home` (resolved via `--home`, then `TULIPROX_HOME`, then binary directory).
|
||||
With `storage_dir: ./data`, the effective directory is `{home}/data`.
|
||||
|
||||
Be aware that different configurations (e.g. user bouquets) along the playlists are stored in this directory.
|
||||
|
||||
@@ -208,7 +230,6 @@ Each provider can optionally enable a `dns` block:
|
||||
- `overrides`: static host -> IP list (used before DNS lookup)
|
||||
- `on_resolve_error`: `keep_last_good` | `fallback_to_hostname` (default: `keep_last_good`)
|
||||
- `on_connect_error`: `try_next_ip` | `rotate_provider_url` (default: `try_next_ip`)
|
||||
- `resolved`: runtime-managed resolved snapshot per host
|
||||
|
||||
Behavior:
|
||||
|
||||
@@ -219,12 +240,12 @@ Behavior:
|
||||
- `keep_vhost=true`: `Host` header keeps `hostname[:port]`.
|
||||
- On connect/timeout errors and `on_connect_error=try_next_ip`, Tuliprox tries the next IP for the same host before rotating provider URL.
|
||||
|
||||
### 1.4.4 `dns.resolved` persistence and visibility
|
||||
### 1.4.4 DNS resolved IP persistence
|
||||
|
||||
- `dns.resolved` is runtime-managed.
|
||||
- It is exposed in `GET /api/v1/config`.
|
||||
- It is also written back to `source.yml` and overwritten on each DNS refresh cycle.
|
||||
- Save endpoints treat `dns.resolved` as managed data and do not accept it as user-controlled input.
|
||||
Resolved IPs are persisted to `{storage_dir}/provider_dns_resolved.json` (not to `source.yml`).
|
||||
This file is written atomically after each DNS refresh cycle and read at startup to seed DNS caches
|
||||
before the background resolver completes its first cycle. On config hot-reloads, DNS caches are
|
||||
carried over from the previous provider instances so that resolved IPs are available immediately.
|
||||
|
||||
### 1.4.5 Provider Failover + DNS Configuration Example
|
||||
|
||||
@@ -253,10 +274,6 @@ provider:
|
||||
overrides:
|
||||
stable.golden-bridge.con:
|
||||
- 203.0.113.10
|
||||
# runtime-managed, written by tuliprox:
|
||||
resolved:
|
||||
hello.provider.me:
|
||||
- 203.0.113.20
|
||||
inputs:
|
||||
- name: my_input
|
||||
type: xtream
|
||||
@@ -477,7 +494,7 @@ metadata_update:
|
||||
|
||||
**Group overview:**
|
||||
|
||||
- `cache_path` (default `metadata`): Directory where TMDB cache files and metadata are stored. Relative paths are resolved against `working_dir`.
|
||||
- `cache_path` (default `metadata`): Directory where TMDB cache files and metadata are stored. Relative paths are resolved against `storage_dir`.
|
||||
Used by all metadata resolution paths (Xtream VOD/Series, local library).
|
||||
- `resolve`: Controls retries for metadata lookup tasks (for example title/date/TMDB resolution).
|
||||
- `probe`: Controls retries and cooldown for technical stream probing tasks.
|
||||
@@ -653,12 +670,25 @@ schedules:
|
||||
- xtream
|
||||
- schedule: "0 0 20 * * * *"
|
||||
type: LibraryScan
|
||||
- schedule: "0 0 4 * * 1 *"
|
||||
type: GeoIpUpdate
|
||||
- schedule: "0 0 4 1 * * *"
|
||||
type: GeoIpUpdate
|
||||
```
|
||||
|
||||
The `type` attribute defines the task to be executed and defaults to `PlaylistUpdate` if omitted. Possible values:
|
||||
|
||||
- `PlaylistUpdate`: Updates the target playlists (optionally filtered by `targets`).
|
||||
- `LibraryScan`: Triggers a scan of the local media library (requires `library` configuration to be enabled).
|
||||
- `GeoIpUpdate`: Downloads and rebuilds the Geo-IP database file (requires `reverse_proxy.geoip.enabled: true`).
|
||||
|
||||
Cron fields for Tuliprox schedules always start with **seconds**:
|
||||
|
||||
```yaml
|
||||
# sec min hour day-of-month month day-of-week year
|
||||
schedule: "0 0 4 * * 1 *" # every Monday at 04:00:00
|
||||
schedule: "0 0 4 1 * * *" # every 1st of the month at 04:00:00
|
||||
```
|
||||
|
||||
At the given times the update is started. Do not start it every second or minute.
|
||||
You could be banned from your server. Twice a day should be enough.
|
||||
@@ -741,7 +771,7 @@ This is useful for players that might time out or error if they receive data and
|
||||
|
||||
#### 1.7.2 `cache`
|
||||
|
||||
LRU-Cache is for resources. If it is `enabled`, the resources/images are persisted in the given `dir`. If the cache size exceeds `size`,
|
||||
LRU-Cache is for resources. If it is `enabled`, the resources/images are persisted in the given `directory`. If the cache size exceeds `size`,
|
||||
In an LRU cache, the least recently used items are evicted to make room for new items if the cache `size`is exceeded.
|
||||
|
||||
#### 1.7.3 `resource_rewrite_disabled`
|
||||
@@ -762,7 +792,7 @@ reverse_proxy:
|
||||
cache:
|
||||
enabled: true
|
||||
size: 1GB
|
||||
dir: ./cache
|
||||
directory: ./cache
|
||||
```
|
||||
|
||||
#### 1.7.4 `rate_limit`
|
||||
@@ -983,13 +1013,17 @@ The encrypted pasword needs to be added manually into the users file.
|
||||
|
||||
```yaml
|
||||
threads: 4
|
||||
working_dir: ./data
|
||||
storage_dir: ./data
|
||||
api:
|
||||
host: localhost
|
||||
port: 8901
|
||||
web_root: ./web
|
||||
```
|
||||
|
||||
Relative paths in this example are resolved under `{home}`.
|
||||
So `storage_dir: ./data` -> `{home}/data` and `web_root: ./web` -> `{home}/web`
|
||||
(`--home` -> `TULIPROX_HOME` -> binary directory).
|
||||
|
||||
### 1.12 `user_access_control`
|
||||
|
||||
The default is `false`.
|
||||
@@ -1463,6 +1497,23 @@ inputs:
|
||||
probe_stream: true
|
||||
```
|
||||
|
||||
#### Input URL Schemes (`inputs[].url`)
|
||||
|
||||
Tuliprox supports the following URL schemes for input sources:
|
||||
|
||||
- `http://...` / `https://...`: Download from remote provider endpoints.
|
||||
- `file://...`: Read from a local file.
|
||||
- `provider://<provider_name>/...`: Resolve the URL via `provider` definitions (supports failover/rotation).
|
||||
- `batch://...`: CSV source for batch aliases (e.g. `batch:///path/file.csv` or `batch://./file.csv`).
|
||||
|
||||
Notes:
|
||||
|
||||
- If `type` is `m3u` or `xtream` and the URL starts with `batch://`, Tuliprox automatically treats it as `m3u_batch` / `xtream_batch`.
|
||||
- For `m3u_batch` and `xtream_batch`, only local CSV sources are supported:
|
||||
- `batch://...` or
|
||||
- plain local filesystem path (absolute/relative).
|
||||
- For batch inputs, `provider://...` and other URI schemes such as `http(s)://...` or `file://...` are rejected.
|
||||
|
||||
Input alias definition for same provider with same content but different credentials.
|
||||
`max_connections` default is unlimited
|
||||
|
||||
@@ -1488,7 +1539,7 @@ sources:
|
||||
|
||||
Input aliases can be defined as batches in csv files with `;` separator.
|
||||
There are 2 batch input types `xtream_batch` and `m3u_batch`.
|
||||
Batch inputs are file-based only: use a local path (`/path/file.csv`, `./file.csv`) or `file://...`.
|
||||
Batch inputs use the `batch://` URL scheme: `batch:///path/file.csv` (absolute) or `batch://./file.csv` (relative).
|
||||
|
||||
#### `XtreamBatch`
|
||||
|
||||
@@ -1496,7 +1547,7 @@ Batch inputs are file-based only: use a local path (`/path/file.csv`, `./file.cs
|
||||
inputs:
|
||||
- type: xtream_batch
|
||||
name: my_provider # Mandatory: used for playlist UUID generation
|
||||
url: 'file:///home/tuliprox/config/my_provider_batch.csv'
|
||||
url: 'batch:///home/tuliprox/config/my_provider_batch.csv'
|
||||
sources:
|
||||
- inputs:
|
||||
- my_provider
|
||||
@@ -1520,7 +1571,7 @@ This is necessary because of playlist uuid generation and assigning same channel
|
||||
```yaml
|
||||
inputs:
|
||||
- type: m3u_batch
|
||||
url: 'file:///home/tuliprox/config/my_provider_batch.csv'
|
||||
url: 'batch:///home/tuliprox/config/my_provider_batch.csv'
|
||||
sources:
|
||||
- inputs:
|
||||
- m3u_batch
|
||||
@@ -2993,25 +3044,30 @@ The provider gives you:
|
||||
- epg_url: `http://fantastic.provider.xyz:8080/xmltv.php?username=tvjunkie&password=junkie.secret`
|
||||
|
||||
To use `tuliprox` you need to create the configuration.
|
||||
The configuration consist of 4 files.
|
||||
The configuration consists of 3 required files and up to 2 optional files.
|
||||
|
||||
Required files:
|
||||
|
||||
- config.yml
|
||||
- source.yml
|
||||
- mapping.yml
|
||||
- api-proxy.yml
|
||||
|
||||
The file `mapping.yml`is optional and only needed if you want to do something linke renaming titles or changing attributes.
|
||||
Optional files:
|
||||
|
||||
- mapping.yml (for advanced transformations like renaming titles or changing attributes)
|
||||
- template.yml (for centralized pattern reuse)
|
||||
|
||||
Lets start with `config.yml`. An example basic configuration is:
|
||||
|
||||
```yaml
|
||||
api: {host: 0.0.0.0, port: 8901, web_root: ./web}
|
||||
working_dir: ./data
|
||||
storage_dir: ./data
|
||||
update_on_boot: true
|
||||
```
|
||||
|
||||
This configuration starts `tuliprox`and listens on the 8901 port. The downloaded playlists are stored inside the `data`-folder in the current working
|
||||
directory.
|
||||
This configuration starts `tuliprox`and listens on the 8901 port. The downloaded playlists are stored inside the `data` folder under the resolved
|
||||
home directory (`--home`, then `TULIPROX_HOME`, then binary directory), i.e. `{home}/data`.
|
||||
`web_root: ./web` resolves to `{home}/web` with the same order.
|
||||
The property `update_on_boot` is optional and can be helpful in the beginning until you have found a working configuration. I prefer to set it to
|
||||
false.
|
||||
|
||||
|
||||
@@ -223,20 +223,24 @@ impl ConfigFile {
|
||||
// ── Parse ────────────────────────────────────────────────────
|
||||
let config_dto = read_config_file(config_file.as_str(), true, true)?;
|
||||
|
||||
let default_mapping_path = utils::get_default_mappings_path(paths.config_path.as_str());
|
||||
let current_mapping_path = paths.mapping_file_path.clone().unwrap_or_else(|| default_mapping_path.clone());
|
||||
let next_mapping_path =
|
||||
config_dto.mapping_path.clone().filter(|path| !path.trim().is_empty()).unwrap_or(default_mapping_path);
|
||||
let current_mapping_path = paths.mapping_file_path.clone().unwrap_or_else(|| {
|
||||
utils::resolve_mapping_file_path(paths.config_path.as_str(), None)
|
||||
});
|
||||
let next_mapping_path = paths.mapping_file_path.clone().unwrap_or_else(|| {
|
||||
utils::resolve_mapping_file_path(paths.config_path.as_str(), config_dto.mapping_path.as_deref())
|
||||
});
|
||||
let mapping_changed = current_mapping_path != next_mapping_path;
|
||||
|
||||
let default_template_path = utils::get_default_templates_path(paths.config_path.as_str());
|
||||
let current_template_path = paths.template_file_path.clone().unwrap_or_else(|| default_template_path.clone());
|
||||
let next_template_path =
|
||||
config_dto.template_path.clone().filter(|path| !path.trim().is_empty()).unwrap_or(default_template_path);
|
||||
let current_template_path = paths.template_file_path.clone().unwrap_or_else(|| {
|
||||
utils::resolve_template_file_path(paths.config_path.as_str(), None)
|
||||
});
|
||||
let next_template_path = paths.template_file_path.clone().unwrap_or_else(|| {
|
||||
utils::resolve_template_file_path(paths.config_path.as_str(), config_dto.template_path.as_deref())
|
||||
});
|
||||
let template_changed = current_template_path != next_template_path;
|
||||
|
||||
let mut config: Config = Config::from(config_dto);
|
||||
config.prepare(paths.config_path.as_str())?;
|
||||
config.prepare(paths.config_path.as_str(), paths.home_path.as_str())?;
|
||||
|
||||
// Compute effective runtime paths for the NEW config before apply.
|
||||
// This ensures prepare-phase reads/validates against the same files that will be active after apply.
|
||||
|
||||
@@ -129,7 +129,7 @@ pub(in crate::api::endpoints) async fn get_playlist_for_custom_provider(
|
||||
let cfg = app_state.app_config.config.load();
|
||||
match cfg_input {
|
||||
Some(input) => {
|
||||
let (result, errors) = match input.input_type {
|
||||
let (result, errors) = match input.get_download_input_type() {
|
||||
InputType::M3u | InputType::M3uBatch => {
|
||||
m3u::download_m3u_playlist(&app_state.app_config, client, &cfg, input).await
|
||||
}
|
||||
|
||||
@@ -34,10 +34,10 @@ async fn scan_library(
|
||||
};
|
||||
|
||||
// Check if Library is enabled
|
||||
let (lib_config, metadata_update_config, working_dir) = {
|
||||
let (lib_config, metadata_update_config, storage_dir) = {
|
||||
let config = app_state.app_config.config.load();
|
||||
match config.library.as_ref() {
|
||||
Some(lib) if lib.enabled => (lib.clone(), config.metadata_update.clone(), config.working_dir.clone()),
|
||||
Some(lib) if lib.enabled => (lib.clone(), config.metadata_update.clone(), config.storage_dir.clone()),
|
||||
_ => {
|
||||
let response = LibraryScanSummary {
|
||||
status: LibraryScanSummaryStatus::Error,
|
||||
@@ -61,7 +61,7 @@ async fn scan_library(
|
||||
lib_config,
|
||||
metadata_update_config,
|
||||
client,
|
||||
LibraryScanTaskOptions { force_rescan: request.force_rescan, message_prefix: "", working_dir },
|
||||
LibraryScanTaskOptions { force_rescan: request.force_rescan, message_prefix: "", storage_dir },
|
||||
permit,
|
||||
);
|
||||
|
||||
@@ -78,7 +78,7 @@ async fn get_library_status(
|
||||
let client = app_state.http_client.load_full().as_ref().clone();
|
||||
// Get statistics from processor
|
||||
let processor =
|
||||
LibraryProcessor::new(config.clone(), config_snapshot.metadata_update.as_ref(), client, &config_snapshot.working_dir);
|
||||
LibraryProcessor::new(config.clone(), config_snapshot.metadata_update.as_ref(), client, &config_snapshot.storage_dir);
|
||||
let entries = processor.get_all_entries().await;
|
||||
|
||||
let movies = entries.iter().filter(|e| e.metadata.is_movie()).count();
|
||||
@@ -90,7 +90,7 @@ async fn get_library_status(
|
||||
movies,
|
||||
series,
|
||||
path: Some(
|
||||
resolve_metadata_storage_path(config_snapshot.metadata_update.as_ref(), &config_snapshot.working_dir)
|
||||
resolve_metadata_storage_path(config_snapshot.metadata_update.as_ref(), &config_snapshot.storage_dir)
|
||||
.to_string_lossy()
|
||||
.to_string(),
|
||||
),
|
||||
|
||||
@@ -10,22 +10,16 @@ use crate::{
|
||||
model::AppState,
|
||||
},
|
||||
auth::validator_admin,
|
||||
model::InputSource,
|
||||
repository::get_geoip_path,
|
||||
utils::{ip_checker::get_ips, request::download_text_content, GeoIp},
|
||||
processing::geoip::{update_geoip_db, GeoIpUpdateError},
|
||||
utils::ip_checker::get_ips,
|
||||
VERSION,
|
||||
};
|
||||
use axum::response::IntoResponse;
|
||||
use log::{error, info};
|
||||
use shared::{
|
||||
model::{default_geoip_url, InputFetchMethod, IpCheckDto, StatusCheck},
|
||||
utils::{concat_path_leading_slash, Internable},
|
||||
};
|
||||
use std::{
|
||||
collections::{BTreeMap, HashMap},
|
||||
io::Cursor,
|
||||
sync::Arc,
|
||||
model::{IpCheckDto, StatusCheck},
|
||||
utils::concat_path_leading_slash,
|
||||
};
|
||||
use std::{collections::BTreeMap, sync::Arc};
|
||||
|
||||
async fn create_ipinfo_check(app_state: &Arc<AppState>) -> Option<(Option<String>, Option<String>)> {
|
||||
let config = app_state.app_config.config.load();
|
||||
@@ -85,63 +79,16 @@ async fn streams(
|
||||
async fn geoip_update(
|
||||
axum::extract::State(app_state): axum::extract::State<Arc<AppState>>,
|
||||
) -> axum::response::Response {
|
||||
let config = app_state.app_config.config.load();
|
||||
if let Some(geoip) = config.reverse_proxy.as_ref().and_then(|r| r.geoip.as_ref()) {
|
||||
if geoip.enabled {
|
||||
let geoip_db_path = &*get_geoip_path(&config.working_dir);
|
||||
let _file_lock = app_state.app_config.file_locks.write_lock(geoip_db_path).await;
|
||||
|
||||
let url = if geoip.url.trim().is_empty() { default_geoip_url() } else { geoip.url.clone() };
|
||||
let input_source = InputSource {
|
||||
name: "GeoIP".intern(),
|
||||
url,
|
||||
provider: None,
|
||||
username: None,
|
||||
password: None,
|
||||
method: InputFetchMethod::GET,
|
||||
headers: HashMap::default(),
|
||||
};
|
||||
return match download_text_content(
|
||||
&app_state.app_config,
|
||||
&app_state.http_client.load(),
|
||||
&input_source,
|
||||
None,
|
||||
None,
|
||||
false,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok((content, _)) => {
|
||||
let reader = Cursor::new(content);
|
||||
let mut geoip = GeoIp::new();
|
||||
let result = {
|
||||
match geoip.import_ipv4_from_csv(reader, geoip_db_path) {
|
||||
Ok(size) => (Some(size), None),
|
||||
Err(err) => (None, Some(err)),
|
||||
}
|
||||
};
|
||||
|
||||
return match result {
|
||||
(Some(_), None) => {
|
||||
info!("GeoIp db updated");
|
||||
app_state.geoip.store(Some(Arc::new(geoip)));
|
||||
axum::http::StatusCode::OK.into_response()
|
||||
}
|
||||
(None, Some(err)) => {
|
||||
error!("Failed to process geoip db: {err}");
|
||||
internal_server_error!()
|
||||
}
|
||||
_ => internal_server_error!(),
|
||||
};
|
||||
}
|
||||
Err(err) => {
|
||||
error!("Failed to download geoip db: {err}");
|
||||
axum::http::StatusCode::BAD_REQUEST.into_response()
|
||||
}
|
||||
};
|
||||
match update_geoip_db(&app_state).await {
|
||||
Ok(()) => axum::http::StatusCode::OK.into_response(),
|
||||
Err(GeoIpUpdateError::Disabled | GeoIpUpdateError::DownloadFailed(_)) => {
|
||||
axum::http::StatusCode::BAD_REQUEST.into_response()
|
||||
}
|
||||
Err(err) => {
|
||||
log::error!("GeoIp update failed: {err}");
|
||||
internal_server_error!()
|
||||
}
|
||||
}
|
||||
axum::http::StatusCode::BAD_REQUEST.into_response()
|
||||
}
|
||||
|
||||
async fn ipinfo(axum::extract::State(app_state): axum::extract::State<Arc<AppState>>) -> axum::response::Response {
|
||||
|
||||
@@ -16,11 +16,11 @@ use axum::{
|
||||
response::IntoResponse,
|
||||
Router,
|
||||
};
|
||||
use log::{error, warn};
|
||||
use log::error;
|
||||
use serde_json::json;
|
||||
use shared::{
|
||||
error::TuliproxError,
|
||||
model::{ApiProxyConfigDto, ConfigDto, ConfigProviderDto, ProviderDnsDto, SourcesConfigDto},
|
||||
model::{ApiProxyConfigDto, ConfigDto, SourcesConfigDto},
|
||||
utils::{
|
||||
HEADER_CONFIG_API_PROXY_REVISION, HEADER_CONFIG_MAIN_REVISION, HEADER_CONFIG_SOURCES_REVISION, HEADER_IF_MATCH,
|
||||
},
|
||||
@@ -28,112 +28,7 @@ use shared::{
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
|
||||
fn inject_provider_dns_resolved(sources_dto: &mut SourcesConfigDto, runtime_sources: &crate::model::SourcesConfig) {
|
||||
let Some(provider_dtos) = sources_dto.provider.as_mut() else {
|
||||
return;
|
||||
};
|
||||
for provider_dto in provider_dtos {
|
||||
let Some(runtime_provider) = runtime_sources.get_provider_by_name(provider_dto.name.as_ref()) else {
|
||||
continue;
|
||||
};
|
||||
let Some(dns_dto) = provider_dto.dns.as_mut() else {
|
||||
continue;
|
||||
};
|
||||
if runtime_provider.get_dns_config().is_none() {
|
||||
dns_dto.resolved = None;
|
||||
continue;
|
||||
}
|
||||
let snapshot = runtime_provider.snapshot_resolved_ordered();
|
||||
dns_dto.resolved = (!snapshot.is_empty()).then_some(snapshot);
|
||||
}
|
||||
}
|
||||
|
||||
fn strip_provider_dns_resolved(sources_dto: &mut SourcesConfigDto) {
|
||||
let Some(provider_dtos) = sources_dto.provider.as_mut() else {
|
||||
return;
|
||||
};
|
||||
for provider_dto in provider_dtos {
|
||||
if let Some(dns_dto) = provider_dto.dns.as_mut() {
|
||||
dns_dto.resolved = None;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn merge_provider_dns_resolved_from_existing(
|
||||
sources_dto: &mut SourcesConfigDto,
|
||||
existing_sources_dto: &SourcesConfigDto,
|
||||
) {
|
||||
let Some(provider_dtos) = sources_dto.provider.as_mut() else {
|
||||
return;
|
||||
};
|
||||
let Some(existing_provider_dtos) = existing_sources_dto.provider.as_ref() else {
|
||||
return;
|
||||
};
|
||||
|
||||
let mut resolved_by_provider_name = std::collections::HashMap::new();
|
||||
for provider in existing_provider_dtos {
|
||||
let Some(dns) = provider.dns.as_ref() else {
|
||||
continue;
|
||||
};
|
||||
let Some(resolved) = dns.resolved.as_ref() else {
|
||||
continue;
|
||||
};
|
||||
if !resolved.is_empty() {
|
||||
resolved_by_provider_name.insert(provider.name.to_string(), resolved.clone());
|
||||
}
|
||||
}
|
||||
|
||||
if resolved_by_provider_name.is_empty() {
|
||||
return;
|
||||
}
|
||||
|
||||
for provider in provider_dtos {
|
||||
let Some(dns) = provider.dns.as_mut() else {
|
||||
continue;
|
||||
};
|
||||
if !dns.enabled {
|
||||
continue;
|
||||
}
|
||||
if dns.resolved.is_some() {
|
||||
continue;
|
||||
}
|
||||
if let Some(resolved) = resolved_by_provider_name.get(provider.name.as_ref()) {
|
||||
dns.resolved = Some(resolved.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn build_existing_sources_from_runtime(runtime_sources: &crate::model::SourcesConfig) -> Option<SourcesConfigDto> {
|
||||
let providers: Vec<ConfigProviderDto> = runtime_sources
|
||||
.provider
|
||||
.iter()
|
||||
.filter_map(|runtime_provider| {
|
||||
let resolved = runtime_provider.snapshot_resolved_ordered();
|
||||
if resolved.is_empty() {
|
||||
return None;
|
||||
}
|
||||
|
||||
Some(ConfigProviderDto {
|
||||
name: runtime_provider.name.clone(),
|
||||
urls: runtime_provider.urls.clone(),
|
||||
dns: Some(ProviderDnsDto {
|
||||
enabled: runtime_provider.get_dns_config().is_some_and(|cfg| cfg.enabled),
|
||||
resolved: Some(resolved),
|
||||
..ProviderDnsDto::default()
|
||||
}),
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
|
||||
if providers.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(SourcesConfigDto {
|
||||
provider: Some(providers),
|
||||
..SourcesConfigDto::default()
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn file_revision_from_bytes(bytes: &[u8]) -> String { blake3::hash(bytes).to_hex().to_string() }
|
||||
|
||||
@@ -273,7 +168,7 @@ async fn save_config_main(
|
||||
async fn save_config_sources(
|
||||
axum::extract::State(app_state): axum::extract::State<Arc<AppState>>,
|
||||
headers: HeaderMap,
|
||||
axum::extract::Json(mut sources): axum::extract::Json<SourcesConfigDto>,
|
||||
axum::extract::Json(sources): axum::extract::Json<SourcesConfigDto>,
|
||||
) -> impl axum::response::IntoResponse + Send {
|
||||
let sources_file_path = {
|
||||
let paths = app_state.app_config.paths.load();
|
||||
@@ -294,27 +189,6 @@ async fn save_config_sources(
|
||||
return response;
|
||||
}
|
||||
|
||||
let existing_sources_dto =
|
||||
match utils::read_sources_file_from_path(Path::new(&sources_file_path), false, false, None).await {
|
||||
Ok(existing) => Some(existing),
|
||||
Err(err) => {
|
||||
warn!("Failed to preload existing source.yml from '{sources_file_path}' before save: {err}");
|
||||
let runtime_sources = app_state.app_config.sources.load();
|
||||
let runtime_fallback = build_existing_sources_from_runtime(runtime_sources.as_ref());
|
||||
if runtime_fallback.is_some() {
|
||||
warn!("Using runtime provider dns snapshot as fallback for preserving dns.resolved");
|
||||
}
|
||||
runtime_fallback
|
||||
}
|
||||
};
|
||||
|
||||
// `dns.resolved` is runtime-managed and must not be accepted from API input,
|
||||
// but existing runtime values should survive UI saves until the next DNS refresh updates them.
|
||||
strip_provider_dns_resolved(&mut sources);
|
||||
if let Some(existing_sources_dto) = existing_sources_dto.as_ref() {
|
||||
merge_provider_dns_resolved_from_existing(&mut sources, existing_sources_dto);
|
||||
}
|
||||
|
||||
let templates_to_persist = match utils::validate_source_config_for_persist(&app_state, &sources).await {
|
||||
Ok(value) => value,
|
||||
Err(err) => {
|
||||
@@ -499,8 +373,6 @@ async fn config(axum::extract::State(app_state): axum::extract::State<Arc<AppSta
|
||||
error!("Failed to prepare users: {err}");
|
||||
internal_server_error!()
|
||||
} else {
|
||||
let runtime_sources = app_state.app_config.sources.load();
|
||||
inject_provider_dns_resolved(&mut app_config.sources, &runtime_sources);
|
||||
let response = axum::response::Json(app_config).into_response();
|
||||
let response = response_with_revision_header(response, HEADER_CONFIG_MAIN_REVISION, &main_revision);
|
||||
let response = response_with_revision_header(response, HEADER_CONFIG_SOURCES_REVISION, &sources_revision);
|
||||
@@ -562,247 +434,9 @@ pub fn v1_api_config_register(router: Router<Arc<AppState>>) -> axum::Router<Arc
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{
|
||||
build_existing_sources_from_runtime, inject_provider_dns_resolved, merge_provider_dns_resolved_from_existing,
|
||||
require_matching_revision, strip_provider_dns_resolved,
|
||||
};
|
||||
use super::require_matching_revision;
|
||||
use axum::http::{HeaderMap, HeaderValue, StatusCode};
|
||||
use crate::model::{ConfigProvider, SourcesConfig};
|
||||
use indexmap::IndexMap;
|
||||
use shared::model::{ConfigProviderDto, DnsScheme, ProviderDnsDto, SourcesConfigDto};
|
||||
use shared::utils::{HEADER_CONFIG_SOURCES_REVISION, HEADER_IF_MATCH};
|
||||
use std::{net::IpAddr, sync::Arc};
|
||||
|
||||
#[test]
|
||||
fn inject_provider_dns_resolved_populates_runtime_snapshot() {
|
||||
let mut dto = SourcesConfigDto {
|
||||
provider: Some(vec![ConfigProviderDto {
|
||||
name: "p1".into(),
|
||||
urls: vec!["http://example.com".into()],
|
||||
dns: Some(ProviderDnsDto {
|
||||
enabled: true,
|
||||
schemes: Some(vec![DnsScheme::Http]),
|
||||
..ProviderDnsDto::default()
|
||||
}),
|
||||
}]),
|
||||
..SourcesConfigDto::default()
|
||||
};
|
||||
|
||||
let runtime_provider = Arc::new(ConfigProvider::from(&ConfigProviderDto {
|
||||
name: "p1".into(),
|
||||
urls: vec!["http://example.com".into()],
|
||||
dns: Some(ProviderDnsDto {
|
||||
enabled: true,
|
||||
schemes: Some(vec![DnsScheme::Http]),
|
||||
..ProviderDnsDto::default()
|
||||
}),
|
||||
}));
|
||||
runtime_provider.store_resolved(
|
||||
"example.com",
|
||||
vec!["203.0.113.10".parse::<IpAddr>().expect("ip parse should work")],
|
||||
);
|
||||
let runtime_sources = SourcesConfig {
|
||||
provider: vec![runtime_provider],
|
||||
..SourcesConfig::default()
|
||||
};
|
||||
|
||||
inject_provider_dns_resolved(&mut dto, &runtime_sources);
|
||||
|
||||
let resolved = dto.provider.as_ref()
|
||||
.and_then(|providers| providers.first())
|
||||
.and_then(|provider| provider.dns.as_ref())
|
||||
.and_then(|dns| dns.resolved.as_ref())
|
||||
.expect("resolved dns snapshot should be present");
|
||||
assert_eq!(
|
||||
resolved.get("example.com"),
|
||||
Some(&vec!["203.0.113.10".parse::<IpAddr>().expect("ip parse should work")])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn inject_provider_dns_resolved_clears_value_when_runtime_dns_disabled() {
|
||||
let mut dto = SourcesConfigDto {
|
||||
provider: Some(vec![ConfigProviderDto {
|
||||
name: "p1".into(),
|
||||
urls: vec!["http://example.com".into()],
|
||||
dns: Some(ProviderDnsDto {
|
||||
enabled: true,
|
||||
resolved: Some(IndexMap::from([(
|
||||
"example.com".to_string(),
|
||||
vec!["203.0.113.10".parse::<IpAddr>().expect("ip parse should work")],
|
||||
)])),
|
||||
..ProviderDnsDto::default()
|
||||
}),
|
||||
}]),
|
||||
..SourcesConfigDto::default()
|
||||
};
|
||||
|
||||
let runtime_provider = Arc::new(ConfigProvider::from(&ConfigProviderDto {
|
||||
name: "p1".into(),
|
||||
urls: vec!["http://example.com".into()],
|
||||
dns: None,
|
||||
}));
|
||||
let runtime_sources = SourcesConfig {
|
||||
provider: vec![runtime_provider],
|
||||
..SourcesConfig::default()
|
||||
};
|
||||
|
||||
inject_provider_dns_resolved(&mut dto, &runtime_sources);
|
||||
|
||||
let resolved = dto.provider.as_ref()
|
||||
.and_then(|providers| providers.first())
|
||||
.and_then(|provider| provider.dns.as_ref())
|
||||
.and_then(|dns| dns.resolved.as_ref());
|
||||
assert!(resolved.is_none(), "resolved output must be empty when runtime dns is disabled");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn strip_provider_dns_resolved_removes_payload_values() {
|
||||
let mut dto = SourcesConfigDto {
|
||||
provider: Some(vec![ConfigProviderDto {
|
||||
name: "p1".into(),
|
||||
urls: vec!["http://example.com".into()],
|
||||
dns: Some(ProviderDnsDto {
|
||||
enabled: true,
|
||||
resolved: Some(IndexMap::from([(
|
||||
"example.com".to_string(),
|
||||
vec!["203.0.113.10".parse::<IpAddr>().expect("ip parse should work")],
|
||||
)])),
|
||||
..ProviderDnsDto::default()
|
||||
}),
|
||||
}]),
|
||||
..SourcesConfigDto::default()
|
||||
};
|
||||
|
||||
strip_provider_dns_resolved(&mut dto);
|
||||
|
||||
let resolved = dto
|
||||
.provider
|
||||
.as_ref()
|
||||
.and_then(|providers| providers.first())
|
||||
.and_then(|provider| provider.dns.as_ref())
|
||||
.and_then(|dns| dns.resolved.as_ref());
|
||||
assert!(resolved.is_none(), "resolved must be stripped from incoming payload");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn merge_provider_dns_resolved_from_existing_preserves_runtime_snapshot() {
|
||||
let mut incoming = SourcesConfigDto {
|
||||
provider: Some(vec![ConfigProviderDto {
|
||||
name: "p1".into(),
|
||||
urls: vec!["http://example.com".into()],
|
||||
dns: Some(ProviderDnsDto {
|
||||
enabled: true,
|
||||
..ProviderDnsDto::default()
|
||||
}),
|
||||
}]),
|
||||
..SourcesConfigDto::default()
|
||||
};
|
||||
let existing = SourcesConfigDto {
|
||||
provider: Some(vec![ConfigProviderDto {
|
||||
name: "p1".into(),
|
||||
urls: vec!["http://example.com".into()],
|
||||
dns: Some(ProviderDnsDto {
|
||||
enabled: true,
|
||||
resolved: Some(IndexMap::from([(
|
||||
"example.com".to_string(),
|
||||
vec!["203.0.113.10".parse::<IpAddr>().expect("ip parse should work")],
|
||||
)])),
|
||||
..ProviderDnsDto::default()
|
||||
}),
|
||||
}]),
|
||||
..SourcesConfigDto::default()
|
||||
};
|
||||
|
||||
merge_provider_dns_resolved_from_existing(&mut incoming, &existing);
|
||||
|
||||
let resolved = incoming
|
||||
.provider
|
||||
.as_ref()
|
||||
.and_then(|providers| providers.first())
|
||||
.and_then(|provider| provider.dns.as_ref())
|
||||
.and_then(|dns| dns.resolved.as_ref())
|
||||
.expect("resolved should be merged from existing source dto");
|
||||
assert_eq!(
|
||||
resolved.get("example.com"),
|
||||
Some(&vec!["203.0.113.10".parse::<IpAddr>().expect("ip parse should work")])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn merge_provider_dns_resolved_from_existing_skips_disabled_dns() {
|
||||
let mut incoming = SourcesConfigDto {
|
||||
provider: Some(vec![ConfigProviderDto {
|
||||
name: "p1".into(),
|
||||
urls: vec!["http://example.com".into()],
|
||||
dns: Some(ProviderDnsDto {
|
||||
enabled: false,
|
||||
..ProviderDnsDto::default()
|
||||
}),
|
||||
}]),
|
||||
..SourcesConfigDto::default()
|
||||
};
|
||||
let existing = SourcesConfigDto {
|
||||
provider: Some(vec![ConfigProviderDto {
|
||||
name: "p1".into(),
|
||||
urls: vec!["http://example.com".into()],
|
||||
dns: Some(ProviderDnsDto {
|
||||
enabled: true,
|
||||
resolved: Some(IndexMap::from([(
|
||||
"example.com".to_string(),
|
||||
vec!["203.0.113.10".parse::<IpAddr>().expect("ip parse should work")],
|
||||
)])),
|
||||
..ProviderDnsDto::default()
|
||||
}),
|
||||
}]),
|
||||
..SourcesConfigDto::default()
|
||||
};
|
||||
|
||||
merge_provider_dns_resolved_from_existing(&mut incoming, &existing);
|
||||
|
||||
let resolved = incoming
|
||||
.provider
|
||||
.as_ref()
|
||||
.and_then(|providers| providers.first())
|
||||
.and_then(|provider| provider.dns.as_ref())
|
||||
.and_then(|dns| dns.resolved.as_ref());
|
||||
assert!(resolved.is_none(), "resolved should not be merged for disabled dns entries");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn build_existing_sources_from_runtime_exports_provider_dns_resolved() {
|
||||
let runtime_provider = Arc::new(ConfigProvider::from(&ConfigProviderDto {
|
||||
name: "p1".into(),
|
||||
urls: vec!["http://example.com".into()],
|
||||
dns: Some(ProviderDnsDto {
|
||||
enabled: true,
|
||||
schemes: Some(vec![DnsScheme::Http]),
|
||||
..ProviderDnsDto::default()
|
||||
}),
|
||||
}));
|
||||
runtime_provider.store_resolved(
|
||||
"example.com",
|
||||
vec!["203.0.113.10".parse::<IpAddr>().expect("ip parse should work")],
|
||||
);
|
||||
let runtime_sources = SourcesConfig {
|
||||
provider: vec![runtime_provider],
|
||||
..SourcesConfig::default()
|
||||
};
|
||||
|
||||
let exported =
|
||||
build_existing_sources_from_runtime(&runtime_sources).expect("runtime fallback dto should exist");
|
||||
let resolved = exported
|
||||
.provider
|
||||
.as_ref()
|
||||
.and_then(|providers| providers.first())
|
||||
.and_then(|provider| provider.dns.as_ref())
|
||||
.and_then(|dns| dns.resolved.as_ref())
|
||||
.expect("resolved should be present in runtime fallback dto");
|
||||
assert_eq!(
|
||||
resolved.get("example.com"),
|
||||
Some(&vec!["203.0.113.10".parse::<IpAddr>().expect("ip parse should work")])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn require_matching_revision_rejects_missing_if_match_header() {
|
||||
|
||||
@@ -10,7 +10,7 @@ use std::sync::Arc;
|
||||
pub(crate) struct LibraryScanTaskOptions {
|
||||
pub force_rescan: bool,
|
||||
pub message_prefix: &'static str,
|
||||
pub working_dir: String,
|
||||
pub storage_dir: String,
|
||||
}
|
||||
|
||||
pub(crate) fn spawn_library_scan(
|
||||
@@ -21,11 +21,11 @@ pub(crate) fn spawn_library_scan(
|
||||
options: LibraryScanTaskOptions,
|
||||
permit: UpdateGuardPermit,
|
||||
) {
|
||||
let LibraryScanTaskOptions { force_rescan, message_prefix, working_dir } = options;
|
||||
let LibraryScanTaskOptions { force_rescan, message_prefix, storage_dir } = options;
|
||||
let prefix = message_prefix.to_string();
|
||||
tokio::spawn(async move {
|
||||
let _permit = permit;
|
||||
let processor = LibraryProcessor::new(lib_config, metadata_update_config.as_ref(), client, &working_dir);
|
||||
let processor = LibraryProcessor::new(lib_config, metadata_update_config.as_ref(), client, &storage_dir);
|
||||
match processor.scan(force_rescan).await {
|
||||
Ok(result) => {
|
||||
info!("{prefix}Library scan completed successfully");
|
||||
|
||||
@@ -204,7 +204,7 @@ async fn create_shared_data(
|
||||
|
||||
let use_geoip = config.is_geoip_enabled();
|
||||
let geoip = if use_geoip {
|
||||
let path = get_geoip_path(&config.working_dir);
|
||||
let path = get_geoip_path(&config.storage_dir);
|
||||
let _file_lock = app_config.file_locks.read_lock(&path).await;
|
||||
match GeoIp::load(&path) {
|
||||
Ok(db) => {
|
||||
|
||||
@@ -708,7 +708,9 @@ mod tests {
|
||||
api_proxy: Arc::new(ArcSwapOption::default()),
|
||||
file_locks: Arc::new(FileLockManager::default()),
|
||||
paths: Arc::new(ArcSwap::from_pointee(ConfigPaths {
|
||||
home_path: String::new(),
|
||||
config_path: String::new(),
|
||||
storage_path: String::new(),
|
||||
config_file_path: String::new(),
|
||||
sources_file_path: String::new(),
|
||||
mapping_file_path: None,
|
||||
|
||||
@@ -266,7 +266,7 @@ fn build_http_client_with_fallback(
|
||||
pub fn create_cache(config: &Config) -> Option<Arc<Mutex<LRUResourceCache>>> {
|
||||
let lru_cache = config.reverse_proxy.as_ref().and_then(|r| r.cache.as_ref()).and_then(|c| {
|
||||
if c.enabled {
|
||||
Some(LRUResourceCache::new(c.size, c.dir.as_str()))
|
||||
Some(LRUResourceCache::new(c.size, c.directory.as_str()))
|
||||
} else {
|
||||
None
|
||||
}
|
||||
@@ -341,20 +341,23 @@ pub struct AppState {
|
||||
|
||||
impl AppState {
|
||||
pub(in crate::api::model) async fn set_config(&self, config: Config) -> Result<UpdateChanges, TuliproxError> {
|
||||
let old_storage_dir = self.app_config.config.load().storage_dir.clone();
|
||||
let changes = self.detect_changes_for_config(&config);
|
||||
config.update_runtime();
|
||||
|
||||
let use_geoip = config.is_geoip_enabled();
|
||||
let working_dir = config.working_dir.clone();
|
||||
let storage_dir = config.storage_dir.clone();
|
||||
|
||||
self.active_users.update_config(&config);
|
||||
self.app_config.set_config(config)?;
|
||||
self.active_provider.update_config(&self.app_config).await;
|
||||
self.update_config().await?;
|
||||
|
||||
if changes.flags.contains(UpdateChangesFlags::Geoip) {
|
||||
let geoip_reload_needed =
|
||||
changes.flags.contains(UpdateChangesFlags::Geoip) || (use_geoip && old_storage_dir != storage_dir);
|
||||
if geoip_reload_needed {
|
||||
let new_geoip = if use_geoip {
|
||||
let path = get_geoip_path(&working_dir);
|
||||
let path = get_geoip_path(&storage_dir);
|
||||
let _file_lock = self.app_config.file_locks.read_lock(&path).await;
|
||||
GeoIp::load(&path).ok().map(Arc::new)
|
||||
} else {
|
||||
@@ -381,7 +384,7 @@ impl AppState {
|
||||
.reverse_proxy
|
||||
.as_ref()
|
||||
.and_then(|r| r.cache.as_ref())
|
||||
.map_or((false, 0, ""), |c| (c.enabled, c.size, c.dir.as_str()));
|
||||
.map_or((false, 0, ""), |c| (c.enabled, c.size, c.directory.as_str()));
|
||||
|
||||
if let Some(cache) = self.cache.load().as_ref() {
|
||||
if enabled {
|
||||
@@ -401,6 +404,22 @@ impl AppState {
|
||||
sources: SourcesConfig,
|
||||
) -> Result<UpdateChanges, TuliproxError> {
|
||||
let changes = self.detect_changes_for_sources(&sources);
|
||||
// Carry over DNS caches from old providers so resolved IPs survive hot-reloads
|
||||
// without waiting for the background resolver or the persisted-file seed.
|
||||
{
|
||||
let old_sources = self.app_config.sources.load();
|
||||
for new_provider in &sources.provider {
|
||||
if let Some(old_provider) = old_sources.get_provider_by_name(&new_provider.name) {
|
||||
if new_provider.get_dns_config().is_some_and(|cfg| cfg.enabled) {
|
||||
for (host, ips) in old_provider.snapshot_resolved() {
|
||||
if !ips.is_empty() && new_provider.dns_cache.ip_count(&host) == 0 {
|
||||
new_provider.dns_cache.store_resolved(&host, ips);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
self.app_config.set_sources(sources)?;
|
||||
self.active_provider.update_config(&self.app_config).await;
|
||||
|
||||
@@ -420,6 +439,9 @@ impl AppState {
|
||||
let old_config = self.app_config.config.load();
|
||||
let changed_schedules =
|
||||
change_detect!(schedules_changed, old_config.schedules.as_ref(), config.schedules.as_ref());
|
||||
let library_enabled = config.library.as_ref().is_some_and(|library| library.enabled);
|
||||
let old_library_enabled = old_config.library.as_ref().is_some_and(|library| library.enabled);
|
||||
let changed_library_enabled = library_enabled != old_library_enabled;
|
||||
let changed_hdhomerun =
|
||||
change_detect!(hdhomerun_changed, old_config.hdhomerun.as_ref(), config.hdhomerun.as_ref());
|
||||
let changed_file_watch =
|
||||
@@ -430,7 +452,7 @@ impl AppState {
|
||||
let geoip_enabled_old = old_config.is_geoip_enabled();
|
||||
|
||||
let mut changes = UpdateChanges { flags: UpdateChangesFlagsSet::new(), targets: None };
|
||||
changes.set_flag_if(changed_schedules, UpdateChangesFlags::Scheduler);
|
||||
changes.set_flag_if(changed_schedules || changed_library_enabled || geoip_enabled != geoip_enabled_old, UpdateChangesFlags::Scheduler);
|
||||
changes.set_flag_if(changed_hdhomerun, UpdateChangesFlags::Hdhomerun);
|
||||
changes.set_flag_if(changed_file_watch, UpdateChangesFlags::FileWatch);
|
||||
changes.set_flag_if(geoip_enabled != geoip_enabled_old, UpdateChangesFlags::Geoip);
|
||||
@@ -582,19 +604,30 @@ fn schedules_changed(a: &[ScheduleConfig], b: &[ScheduleConfig]) -> bool {
|
||||
if a.len() != b.len() {
|
||||
return true;
|
||||
}
|
||||
let mut used = vec![false; b.len()];
|
||||
|
||||
for schedule in a {
|
||||
let Some(found) = b.iter().find(|&s| s.schedule == schedule.schedule) else {
|
||||
let Some(found_idx) = b.iter().enumerate().find_map(|(idx, candidate)| {
|
||||
if used[idx]
|
||||
|| candidate.schedule != schedule.schedule
|
||||
|| candidate.task_type != schedule.task_type
|
||||
{
|
||||
return None;
|
||||
}
|
||||
let targets_match = match (schedule.targets.as_ref(), candidate.targets.as_ref()) {
|
||||
(None, None) => true,
|
||||
(Some(_), None) | (None, Some(_)) => false,
|
||||
(Some(a_targets), Some(b_targets)) => small_vecs_equal_unordered(a_targets, b_targets),
|
||||
};
|
||||
if targets_match {
|
||||
Some(idx)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}) else {
|
||||
return true;
|
||||
};
|
||||
match (schedule.targets.as_ref(), found.targets.as_ref()) {
|
||||
(None, None) => {}
|
||||
(Some(_), None) | (None, Some(_)) => return true,
|
||||
(Some(a_targets), Some(b_targets)) => {
|
||||
if !small_vecs_equal_unordered(a_targets, b_targets) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
used[found_idx] = true;
|
||||
}
|
||||
false
|
||||
}
|
||||
@@ -635,7 +668,9 @@ pub struct HdHomerunAppState {
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{should_use_manual_redirect_for_proxy, should_use_manual_redirects_for_env_vars};
|
||||
use super::{schedules_changed, should_use_manual_redirect_for_proxy, should_use_manual_redirects_for_env_vars};
|
||||
use crate::model::ScheduleConfig;
|
||||
use shared::model::ScheduleTaskType;
|
||||
|
||||
#[test]
|
||||
fn should_use_manual_redirect_for_proxy_only_http_or_https() {
|
||||
@@ -672,4 +707,48 @@ mod tests {
|
||||
"http://localhost".to_string(),
|
||||
)]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn schedules_changed_detects_task_type_changes() {
|
||||
let a = vec![ScheduleConfig {
|
||||
schedule: "0 0 3 * * * *".to_string(),
|
||||
task_type: ScheduleTaskType::PlaylistUpdate,
|
||||
targets: None,
|
||||
}];
|
||||
let b = vec![ScheduleConfig {
|
||||
schedule: "0 0 3 * * * *".to_string(),
|
||||
task_type: ScheduleTaskType::GeoIpUpdate,
|
||||
targets: None,
|
||||
}];
|
||||
assert!(schedules_changed(&a, &b));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn schedules_changed_treats_same_entries_as_unchanged() {
|
||||
let a = vec![
|
||||
ScheduleConfig {
|
||||
schedule: "0 0 3 * * * *".to_string(),
|
||||
task_type: ScheduleTaskType::GeoIpUpdate,
|
||||
targets: None,
|
||||
},
|
||||
ScheduleConfig {
|
||||
schedule: "0 0 8 * * * *".to_string(),
|
||||
task_type: ScheduleTaskType::PlaylistUpdate,
|
||||
targets: Some(vec!["a".to_string(), "b".to_string()]),
|
||||
},
|
||||
];
|
||||
let b = vec![
|
||||
ScheduleConfig {
|
||||
schedule: "0 0 8 * * * *".to_string(),
|
||||
task_type: ScheduleTaskType::PlaylistUpdate,
|
||||
targets: Some(vec!["b".to_string(), "a".to_string()]),
|
||||
},
|
||||
ScheduleConfig {
|
||||
schedule: "0 0 3 * * * *".to_string(),
|
||||
task_type: ScheduleTaskType::GeoIpUpdate,
|
||||
targets: None,
|
||||
},
|
||||
];
|
||||
assert!(!schedules_changed(&a, &b));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -596,6 +596,7 @@ impl MetadataUpdateManager {
|
||||
/// * `task` - The task to process
|
||||
#[allow(clippy::too_many_lines)]
|
||||
pub async fn queue_task(&self, input_name: Arc<str>, task: UpdateTask) {
|
||||
debug!("[Task] Queuing task for input {input_name}: {task}");
|
||||
// Read app state once and reuse for worker creation when needed.
|
||||
let app_state_weak = {
|
||||
let guard = self.app_state.lock().await;
|
||||
@@ -747,8 +748,12 @@ impl MetadataUpdateManager {
|
||||
}
|
||||
} else {
|
||||
let mut existing = entry.task.lock();
|
||||
let before = format!("{existing}");
|
||||
if Self::merge_task_payload(&mut existing, task_to_submit) {
|
||||
entry.generation.fetch_add(1, Ordering::Relaxed);
|
||||
debug!("[Task] Merged task for input {input_name}: before={before}, after={existing}");
|
||||
} else {
|
||||
debug!("[Task] Task already pending for input {input_name} (no merge needed): {existing}");
|
||||
}
|
||||
return SubmitTaskResult::QueuedOrMerged;
|
||||
}
|
||||
@@ -987,8 +992,8 @@ impl InputWorker {
|
||||
let mut task_for_execution = current_task.clone();
|
||||
|
||||
if Self::is_resolve_task(&task_for_execution) && self.resolve_exhausted.contains_key(¤t_key) {
|
||||
debug_if_enabled!(
|
||||
"Skipping exhausted resolve task for input {}: {} (reset window: {}s)",
|
||||
debug!(
|
||||
"[Metadata-Task] Skipping task (resolve exhausted) for input {}: {} (reset window: {}s)",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
runtime_settings.resolve_exhaustion_reset_gap_secs
|
||||
@@ -1003,10 +1008,8 @@ impl InputWorker {
|
||||
if !skip_execution
|
||||
&& self.should_skip_recent_no_change_task(¤t_key, &task_for_execution, &runtime_settings)
|
||||
{
|
||||
debug_if_enabled!(
|
||||
"Skipping recently-resolved no-change task for input {}: {}",
|
||||
input_name,
|
||||
task_for_execution
|
||||
debug!(
|
||||
"[Metadata-Task] Skipping task (recently completed with no changes) for input {input_name}: {task_for_execution}",
|
||||
);
|
||||
remove_current_task = true;
|
||||
skip_execution = true;
|
||||
@@ -1019,19 +1022,21 @@ impl InputWorker {
|
||||
if let Some(cooldown_until_ts) = tmdb_state.cooldown_until_ts {
|
||||
if now_ts < cooldown_until_ts {
|
||||
if let Some(stripped_task) = Self::strip_tmdb_reasons(&task_for_execution) {
|
||||
debug_if_enabled!(
|
||||
"TMDB cooldown active for input {}: {} (cooldown_until={}), continuing with non-TMDB reasons",
|
||||
debug!(
|
||||
"[Task] TMDB cooldown active for input {}: {}, continuing with non-TMDB reasons (cooldown_until={}, remaining={}s)",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
cooldown_until_ts
|
||||
cooldown_until_ts,
|
||||
cooldown_until_ts.saturating_sub(now_ts)
|
||||
);
|
||||
task_for_execution = stripped_task;
|
||||
} else {
|
||||
debug_if_enabled!(
|
||||
"Skipping TMDB-only task in cooldown for input {}: {} (cooldown_until={})",
|
||||
debug!(
|
||||
"[Metadata-Task] Skipping task (TMDB-only in cooldown) for input {}: {} (cooldown_until={}, remaining={}s)",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
cooldown_until_ts
|
||||
cooldown_until_ts,
|
||||
cooldown_until_ts.saturating_sub(now_ts)
|
||||
);
|
||||
self.scheduled_requeues.remove(¤t_key);
|
||||
remove_current_task = true;
|
||||
@@ -1053,11 +1058,12 @@ impl InputWorker {
|
||||
if active_retry_domain == RetryDomain::Probe {
|
||||
if let Some(cooldown_until_ts) = active_state.cooldown_until_ts {
|
||||
if now_ts < cooldown_until_ts {
|
||||
debug_if_enabled!(
|
||||
"Skipping probe task in cooldown for input {}: {} (cooldown_until={})",
|
||||
debug!(
|
||||
"[Metadata-Task] Skipping task (probe cooldown) for input {}: {} (cooldown_until={}, remaining={}s)",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
cooldown_until_ts
|
||||
cooldown_until_ts,
|
||||
cooldown_until_ts.saturating_sub(now_ts)
|
||||
);
|
||||
self.scheduled_requeues.remove(¤t_key);
|
||||
remove_current_task = true;
|
||||
@@ -1069,6 +1075,14 @@ impl InputWorker {
|
||||
}
|
||||
|
||||
if !skip_execution && active_state.next_allowed_at_ts > now_ts {
|
||||
debug!(
|
||||
"[Task] Deferring task (retry backoff) for input {}: {} (next_allowed_at={}, wait={}s, attempts={})",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
active_state.next_allowed_at_ts,
|
||||
active_state.next_allowed_at_ts.saturating_sub(now_ts),
|
||||
active_state.attempts
|
||||
);
|
||||
schedule_requeue_at_ts = Some(active_state.next_allowed_at_ts);
|
||||
skip_execution = true;
|
||||
}
|
||||
@@ -1103,6 +1117,12 @@ impl InputWorker {
|
||||
}
|
||||
|
||||
if !skip_execution {
|
||||
debug!(
|
||||
"[Task] Executing task for input {}: {} (retry_domain={:?})",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
Self::retry_domain_for_task(&task_for_execution)
|
||||
);
|
||||
let task_result = {
|
||||
let Some(_pause_guard) = self.wait_for_update_pause_window().await else {
|
||||
break;
|
||||
@@ -1129,7 +1149,7 @@ impl InputWorker {
|
||||
Self::should_trigger_playlist_update_for_task(&task_for_execution, task_outcome.task_changed);
|
||||
cycle_had_changes |= trigger_playlist_update;
|
||||
debug!(
|
||||
"Processed metadata task for input {input_name}: {task_for_execution} (changed={}, trigger_playlist_update={}, tmdb_pending={})",
|
||||
"[Metadata-Task] Task succeeded for input {input_name}: {task_for_execution} (changed={}, trigger_playlist_update={}, tmdb_pending={})",
|
||||
task_outcome.task_changed,
|
||||
trigger_playlist_update,
|
||||
task_outcome.tmdb_pending
|
||||
@@ -1151,11 +1171,12 @@ impl InputWorker {
|
||||
tmdb_state.cooldown_until_ts = Some(cooldown_until_ts);
|
||||
tmdb_state.last_error =
|
||||
Some("TMDB lookup completed without matching result".to_string());
|
||||
debug_if_enabled!(
|
||||
"TMDB resolve produced no match for input {}: {} (cooldown_until={})",
|
||||
debug!(
|
||||
"[Metadata-Task] TMDB resolve produced no match (existing retry state), entering cooldown for input {}: {} (cooldown_until={}, cooldown_duration={}s)",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
cooldown_until_ts
|
||||
cooldown_until_ts,
|
||||
runtime_settings.tmdb_cooldown_secs
|
||||
);
|
||||
} else {
|
||||
state_bundle.clear_domain(RetryDomain::Tmdb);
|
||||
@@ -1183,11 +1204,12 @@ impl InputWorker {
|
||||
};
|
||||
self.retry_states.insert(current_key.clone(), state_bundle.clone());
|
||||
state_after_success = Some(state_bundle);
|
||||
debug_if_enabled!(
|
||||
"TMDB resolve produced no match for input {}: {} (cooldown_until={})",
|
||||
debug!(
|
||||
"[Metadata-Task] TMDB resolve produced no match (new retry state), entering cooldown for input {}: {} (cooldown_until={}, cooldown_duration={}s)",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
cooldown_until_ts
|
||||
cooldown_until_ts,
|
||||
runtime_settings.tmdb_cooldown_secs
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1207,6 +1229,10 @@ impl InputWorker {
|
||||
&& !task_outcome.tmdb_pending
|
||||
{
|
||||
let reasons = Self::task_reason(&task_for_execution);
|
||||
debug!(
|
||||
"[Task] Caching no-change result for input {}: {} (reasons={}, ttl={}s)",
|
||||
input_name, task_for_execution, reasons, runtime_settings.no_change_cache_ttl_secs
|
||||
);
|
||||
self.recently_completed_no_change.insert(current_key.clone(), (Instant::now(), reasons));
|
||||
} else {
|
||||
self.recently_completed_no_change.remove(¤t_key);
|
||||
@@ -1237,6 +1263,10 @@ impl InputWorker {
|
||||
}
|
||||
Err(e) => {
|
||||
if Self::is_permanent_not_found_error(&e.message) {
|
||||
debug!(
|
||||
"[Task] Task failed with permanent not-found for input {}: {} (error={})",
|
||||
input_name, task_for_execution, e.message
|
||||
);
|
||||
let retry_domain = Self::retry_domain_for_task(&task_for_execution);
|
||||
self.scheduled_requeues.remove(¤t_key);
|
||||
if retry_domain == RetryDomain::Probe {
|
||||
@@ -1253,11 +1283,12 @@ impl InputWorker {
|
||||
};
|
||||
metadata_persist_state = Some(Some(state_bundle_after_update));
|
||||
remove_current_task = true;
|
||||
debug_if_enabled!(
|
||||
"Probe task entered cooldown after permanent not-found for input {}: {} (cooldown_until={})",
|
||||
debug!(
|
||||
"[Metadata-Task] Probe task entering cooldown after permanent not-found for input {}: {} (cooldown_until={}, cooldown_duration={}s)",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
cooldown_until_ts
|
||||
cooldown_until_ts,
|
||||
runtime_settings.probe_cooldown_secs
|
||||
);
|
||||
} else {
|
||||
self.resolve_exhausted.insert(current_key.clone(), now_ts);
|
||||
@@ -1277,10 +1308,11 @@ impl InputWorker {
|
||||
}
|
||||
metadata_persist_state = Some(state_after_clear);
|
||||
remove_current_task = true;
|
||||
debug_if_enabled!(
|
||||
"Resolve task marked exhausted after permanent not-found for input {}: {}",
|
||||
debug!(
|
||||
"[Metadata-Task] Resolve task marked exhausted after permanent not-found for input {}: {} (error={})",
|
||||
input_name,
|
||||
task_for_execution
|
||||
task_for_execution,
|
||||
e.message
|
||||
);
|
||||
}
|
||||
} else if Self::is_transient_worker_error(&e.message) {
|
||||
@@ -1293,8 +1325,8 @@ impl InputWorker {
|
||||
Self::compute_retry_delay_secs(current_task.delay(), &runtime_settings);
|
||||
let retry_delay_i64 = i64::try_from(retry_delay_secs).unwrap_or(i64::MAX);
|
||||
schedule_requeue_at_ts = Some(now_ts.saturating_add(retry_delay_i64));
|
||||
debug_if_enabled!(
|
||||
"Transient task deferral for input {}: {} (retry_in={}s, err={})",
|
||||
debug!(
|
||||
"[Metadata-Task] Task deferred (transient error) for input {}: {} (retry_in={}s, error={})",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
retry_delay_secs,
|
||||
@@ -1350,12 +1382,14 @@ impl InputWorker {
|
||||
let cooldown_until = state_after_update
|
||||
.cooldown_until_ts
|
||||
.map_or_else(|| "none".to_string(), |ts| ts.to_string());
|
||||
debug_if_enabled!(
|
||||
"Probe task exhausted for input {}: {} (attempts={}, cooldown_until={})",
|
||||
debug!(
|
||||
"Metadata-[Task] Probe task exhausted (max attempts reached) for input {}: {} (attempts={}/{}, cooldown_until={}, error={})",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
state_after_update.attempts,
|
||||
cooldown_until
|
||||
max_attempts,
|
||||
cooldown_until,
|
||||
e.message
|
||||
);
|
||||
} else {
|
||||
self.resolve_exhausted.insert(current_key.clone(), now_ts);
|
||||
@@ -1375,22 +1409,26 @@ impl InputWorker {
|
||||
}
|
||||
metadata_persist_state = Some(state_after_clear);
|
||||
remove_current_task = true;
|
||||
debug_if_enabled!(
|
||||
"Resolve task exhausted for input {}: {} (attempts={})",
|
||||
debug!(
|
||||
"[Metadata-Task] Resolve task exhausted (max attempts reached) for input {}: {} (attempts={}/{}, error={})",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
attempts
|
||||
attempts,
|
||||
max_attempts,
|
||||
e.message
|
||||
);
|
||||
}
|
||||
} else {
|
||||
schedule_requeue_at_ts = Some(state_after_update.next_allowed_at_ts);
|
||||
metadata_persist_state = Some(Some(state_bundle_after_update));
|
||||
debug_if_enabled!(
|
||||
"Task failed for input {}, scheduling retry: {} (attempt={}, next_allowed_at={}, err={})",
|
||||
debug!(
|
||||
"[Metadata-Task] Task failed, scheduling retry for input {}: {} (attempt={}/{}, next_allowed_at={}, backoff={}s, error={})",
|
||||
input_name,
|
||||
task_for_execution,
|
||||
attempts,
|
||||
max_attempts,
|
||||
state_after_update.next_allowed_at_ts,
|
||||
state_after_update.next_allowed_at_ts.saturating_sub(now_ts),
|
||||
e.message
|
||||
);
|
||||
}
|
||||
@@ -1413,6 +1451,13 @@ impl InputWorker {
|
||||
}
|
||||
|
||||
if let Some(retry_at_ts) = schedule_requeue_at_ts {
|
||||
debug!(
|
||||
"[Task] Scheduling requeue for input {}: {:?} (retry_at_ts={}, in={}s)",
|
||||
input_name,
|
||||
current_key,
|
||||
retry_at_ts,
|
||||
retry_at_ts.saturating_sub(chrono::Utc::now().timestamp())
|
||||
);
|
||||
self.schedule_requeue_at(current_key.clone(), retry_at_ts);
|
||||
}
|
||||
|
||||
@@ -1510,8 +1555,8 @@ impl InputWorker {
|
||||
return;
|
||||
};
|
||||
|
||||
let working_dir = app_state.app_config.config.load().working_dir.clone();
|
||||
let Ok(storage_path) = get_input_storage_path(input_name, &working_dir).await else {
|
||||
let storage_dir = app_state.app_config.config.load().storage_dir.clone();
|
||||
let Ok(storage_path) = get_input_storage_path(input_name, &storage_dir).await else {
|
||||
warn!("Could not resolve storage path for metadata retry state on input {input_name}");
|
||||
self.metadata_retry_load_retry_at_ts =
|
||||
Some(now_ts.saturating_add(runtime_settings.metadata_retry_load_retry_delay_secs));
|
||||
@@ -2026,7 +2071,7 @@ impl InputWorker {
|
||||
|
||||
let Some(app_state) = app_state_weak.and_then(Weak::upgrade) else { return };
|
||||
let app_config = &app_state.app_config;
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let cfg = app_config.config.load();
|
||||
let vod_updates = batch_buffer.take_vod_updates();
|
||||
let series_updates = batch_buffer.take_series_updates();
|
||||
let live_updates = batch_buffer.take_live_updates();
|
||||
@@ -2035,7 +2080,7 @@ impl InputWorker {
|
||||
return;
|
||||
}
|
||||
|
||||
if let Ok(storage_path) = get_input_storage_path(input_name, working_dir).await {
|
||||
if let Ok(storage_path) = get_input_storage_path(input_name, &cfg.storage_dir).await {
|
||||
if !vod_updates.is_empty() {
|
||||
let mut updates: Vec<(u32, VideoStreamProperties)> = Vec::with_capacity(vod_updates.len());
|
||||
for (id, props) in &vod_updates {
|
||||
@@ -2531,8 +2576,8 @@ impl InputWorker {
|
||||
}
|
||||
|
||||
if let std::collections::hash_map::Entry::Vacant(entry) = db_handles.entry(cluster) {
|
||||
let working_dir = &app_state.app_config.config.load().working_dir;
|
||||
if let Ok(storage_path) = get_input_storage_path(input_name, working_dir).await {
|
||||
let cfg = app_state.app_config.config.load();
|
||||
if let Ok(storage_path) = get_input_storage_path(input_name, &cfg.storage_dir).await {
|
||||
let file_path = xtream_get_file_path(&storage_path, cluster);
|
||||
if file_path.exists() {
|
||||
let lock = app_state.app_config.file_locks.read_lock(&file_path).await;
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
use crate::api::model::AppState;
|
||||
use crate::model::ConfigProvider;
|
||||
use crate::utils::read_sources_file_from_path;
|
||||
use crate::repository::{
|
||||
load_persisted_dns_resolved, next_dns_writer_generation, prune_persisted_dns_resolved_to_runtime,
|
||||
queue_provider_resolved_snapshot, spawn_dns_resolved_writer, DnsResolvedWriteTx,
|
||||
};
|
||||
use log::{debug, warn};
|
||||
use shared::model::{DnsPrefer, OnResolveErrorPolicy, SourcesConfigDto};
|
||||
use shared::model::{DnsPrefer, OnResolveErrorPolicy};
|
||||
use std::collections::HashSet;
|
||||
use std::io;
|
||||
use std::net::IpAddr;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
use tokio::fs;
|
||||
use tokio::net::lookup_host;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
|
||||
@@ -105,129 +105,12 @@ async fn resolve_provider(provider: &Arc<ConfigProvider>) -> ProviderResolveStat
|
||||
stats
|
||||
}
|
||||
|
||||
fn serialize_sources_for_persist(sources: &SourcesConfigDto) -> io::Result<String> {
|
||||
let mut serialized = String::new();
|
||||
let options = serde_saphyr::SerializerOptions {
|
||||
prefer_block_scalars: false,
|
||||
..Default::default()
|
||||
};
|
||||
serde_saphyr::to_fmt_writer_with_options(&mut serialized, sources, options)
|
||||
.map_err(|err| io::Error::other(format!("Could not serialize source.yml: {err}")))?;
|
||||
Ok(serialized)
|
||||
}
|
||||
|
||||
async fn write_sources_file_force(path: &Path, sources: &SourcesConfigDto) -> io::Result<()> {
|
||||
let serialized = serialize_sources_for_persist(sources)?;
|
||||
let parent_dir = path.parent().ok_or_else(|| {
|
||||
io::Error::other(format!(
|
||||
"Could not write source.yml '{}': missing parent directory",
|
||||
path.display()
|
||||
))
|
||||
})?;
|
||||
let dest_file_name = path.file_name().and_then(|s| s.to_str()).unwrap_or("source.yml");
|
||||
let mut tmp_path = parent_dir.to_path_buf();
|
||||
tmp_path.push(format!(
|
||||
".{dest_file_name}.tmp-{}-{}",
|
||||
std::process::id(),
|
||||
chrono::Local::now().timestamp_nanos_opt().unwrap_or_default()
|
||||
));
|
||||
|
||||
fs::write(&tmp_path, serialized).await?;
|
||||
match fs::rename(&tmp_path, path).await {
|
||||
Ok(()) => Ok(()),
|
||||
Err(err) => {
|
||||
#[cfg(windows)]
|
||||
{
|
||||
// Try to rename again after removing destination (Windows often needs this)
|
||||
if let Ok(()) = fs::remove_file(path).await {
|
||||
if fs::rename(&tmp_path, path).await.is_ok() {
|
||||
return Ok(());
|
||||
}
|
||||
// Rename still failed after removing dest - fall through to clean up
|
||||
}
|
||||
}
|
||||
let _ = fs::remove_file(&tmp_path).await;
|
||||
Err(io::Error::other(format!(
|
||||
"Could not replace '{}' with '{}': {err}",
|
||||
path.display(),
|
||||
tmp_path.display()
|
||||
)))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn persist_provider_resolved_to_source_file(app_state: &Arc<AppState>, provider: &Arc<ConfigProvider>) {
|
||||
let source_file = {
|
||||
let paths = app_state.app_config.paths.load();
|
||||
paths.sources_file_path.clone()
|
||||
};
|
||||
let source_path = PathBuf::from(&source_file);
|
||||
let _lock = app_state.app_config.file_locks.write_lock(&source_path).await;
|
||||
|
||||
let mut sources_dto = match read_sources_file_from_path(&source_path, false, false, None).await {
|
||||
Ok(dto) => dto,
|
||||
Err(err) => {
|
||||
warn!(
|
||||
"Provider dns '{}' failed to read source.yml '{}': {err}",
|
||||
provider.name,
|
||||
source_path.display()
|
||||
);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let Some(provider_dtos) = sources_dto.provider.as_mut() else {
|
||||
debug!(
|
||||
"Provider dns '{}' source.yml '{}' has no provider section to persist resolved values",
|
||||
provider.name,
|
||||
source_path.display()
|
||||
);
|
||||
return;
|
||||
};
|
||||
|
||||
let Some(provider_dto) = provider_dtos.iter_mut().find(|dto| dto.name.as_ref() == provider.name.as_ref()) else {
|
||||
warn!(
|
||||
"Provider dns '{}' not found in source.yml '{}', cannot persist resolved values",
|
||||
provider.name,
|
||||
source_path.display()
|
||||
);
|
||||
return;
|
||||
};
|
||||
|
||||
let Some(dns_dto) = provider_dto.dns.as_mut() else {
|
||||
debug!(
|
||||
"Provider dns '{}' has no dns section in source.yml '{}', skipping resolved persist",
|
||||
provider.name,
|
||||
source_path.display()
|
||||
);
|
||||
return;
|
||||
};
|
||||
|
||||
let resolved_hosts = {
|
||||
let snapshot = provider.snapshot_resolved_ordered();
|
||||
dns_dto.resolved = (!snapshot.is_empty()).then_some(snapshot);
|
||||
dns_dto.resolved.as_ref().map_or(0, indexmap::IndexMap::len)
|
||||
};
|
||||
|
||||
match write_sources_file_force(&source_path, &sources_dto).await {
|
||||
Ok(()) => {
|
||||
debug!(
|
||||
"Provider dns '{}' persisted dns.resolved to '{}' (hosts={resolved_hosts})",
|
||||
provider.name,
|
||||
source_path.display()
|
||||
);
|
||||
}
|
||||
Err(err) => {
|
||||
warn!(
|
||||
"Provider dns '{}' failed to persist dns.resolved to '{}': {err}",
|
||||
provider.name,
|
||||
source_path.display()
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn spawn_provider_dns_task(app_state: Arc<AppState>, provider_name: Arc<str>, cancel: CancellationToken) {
|
||||
fn spawn_provider_dns_task(
|
||||
app_state: Arc<AppState>,
|
||||
provider_name: Arc<str>,
|
||||
cancel: CancellationToken,
|
||||
writer_tx: DnsResolvedWriteTx,
|
||||
) {
|
||||
tokio::spawn(async move {
|
||||
let mut refresh_secs = 300_u64;
|
||||
{
|
||||
@@ -260,7 +143,7 @@ fn spawn_provider_dns_task(app_state: Arc<AppState>, provider_name: Arc<str>, ca
|
||||
};
|
||||
refresh_secs = provider.get_dns_config().map_or(300, |dns| dns.refresh_secs.max(10));
|
||||
let stats = resolve_provider(&provider).await;
|
||||
persist_provider_resolved_to_source_file(&app_state, &provider).await;
|
||||
queue_provider_resolved_snapshot(&writer_tx, &provider).await;
|
||||
let cache_hosts = provider.snapshot_resolved().len();
|
||||
debug!(
|
||||
"Provider dns tick '{}' finished: total_hosts={} resolved={} overridden={} empty={} failed={} cache_hosts={} elapsed_ms={}",
|
||||
@@ -282,6 +165,7 @@ fn spawn_provider_dns_task(app_state: Arc<AppState>, provider_name: Arc<str>, ca
|
||||
}
|
||||
|
||||
pub fn exec_provider_dns(app_state: &Arc<AppState>, cancel: &CancellationToken) {
|
||||
let generation = next_dns_writer_generation();
|
||||
let sources = app_state.app_config.sources.load();
|
||||
let provider_names: Vec<_> = sources
|
||||
.provider
|
||||
@@ -292,13 +176,33 @@ pub fn exec_provider_dns(app_state: &Arc<AppState>, cancel: &CancellationToken)
|
||||
drop(sources);
|
||||
|
||||
if provider_names.is_empty() {
|
||||
let app_for_prune = Arc::clone(app_state);
|
||||
tokio::spawn(async move {
|
||||
prune_persisted_dns_resolved_to_runtime(&app_for_prune).await;
|
||||
});
|
||||
debug!("Provider dns manager: no enabled providers found");
|
||||
return;
|
||||
}
|
||||
|
||||
debug!("Provider dns manager: starting {} provider task(s)", provider_names.len());
|
||||
let app_for_start = Arc::clone(app_state);
|
||||
let cancel_for_start = cancel.clone();
|
||||
tokio::spawn(async move {
|
||||
// Seed runtime caches first, then start writer/tasks to avoid startup races.
|
||||
load_persisted_dns_resolved(&app_for_start).await;
|
||||
if cancel_for_start.is_cancelled() {
|
||||
return;
|
||||
}
|
||||
|
||||
for provider_name in provider_names {
|
||||
spawn_provider_dns_task(Arc::clone(app_state), provider_name, cancel.clone());
|
||||
}
|
||||
let writer_tx = spawn_dns_resolved_writer(Arc::clone(&app_for_start), cancel_for_start.clone(), generation);
|
||||
debug!("Provider dns manager: starting {} provider task(s)", provider_names.len());
|
||||
|
||||
for provider_name in provider_names {
|
||||
spawn_provider_dns_task(
|
||||
Arc::clone(&app_for_start),
|
||||
provider_name,
|
||||
cancel_for_start.clone(),
|
||||
writer_tx.clone(),
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@@ -25,7 +25,6 @@ use reqwest::{
|
||||
};
|
||||
use shared::{
|
||||
create_bitset,
|
||||
model::{PlaylistItemType, DEFAULT_USER_AGENT},
|
||||
utils::{filter_request_header, sanitize_sensitive_info},
|
||||
};
|
||||
use std::{
|
||||
@@ -38,6 +37,8 @@ use std::{
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
use url::Url;
|
||||
use shared::model::PlaylistItemType;
|
||||
use shared::utils::DEFAULT_USER_AGENT;
|
||||
|
||||
const RETRY_SECONDS: u64 = 5;
|
||||
const ERR_MAX_RETRY_COUNT: u32 = 5;
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
use crate::{
|
||||
api::{library_scan::{spawn_library_scan, LibraryScanTaskOptions}, model::AppState},
|
||||
api::{
|
||||
library_scan::{spawn_library_scan, LibraryScanTaskOptions},
|
||||
model::AppState,
|
||||
},
|
||||
model::{AppConfig, ProcessTargets, ScheduleConfig},
|
||||
processing::geoip::{update_geoip_db, GeoIpUpdateError},
|
||||
processing::processor::exec_processing,
|
||||
utils::exit,
|
||||
};
|
||||
@@ -39,6 +43,24 @@ pub fn exec_scheduler(
|
||||
let schedules: Vec<ScheduleConfig> =
|
||||
if let Some(schedules) = &config.schedules { schedules.clone() } else { vec![] };
|
||||
for schedule in schedules {
|
||||
let task_enabled = match schedule.task_type {
|
||||
ScheduleTaskType::PlaylistUpdate => true,
|
||||
ScheduleTaskType::LibraryScan => config.library.as_ref().is_some_and(|library| library.enabled),
|
||||
ScheduleTaskType::GeoIpUpdate => config
|
||||
.reverse_proxy
|
||||
.as_ref()
|
||||
.and_then(|reverse_proxy| reverse_proxy.geoip.as_ref())
|
||||
.is_some_and(|geoip| geoip.enabled),
|
||||
};
|
||||
if !task_enabled {
|
||||
log::info!(
|
||||
"Skipping disabled scheduled task {:?} ({})",
|
||||
schedule.task_type,
|
||||
schedule.schedule
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
let expression = schedule.schedule.clone();
|
||||
let task_type = schedule.task_type;
|
||||
let exec_targets = get_process_targets(cfg, targets, schedule.targets.as_ref());
|
||||
@@ -75,6 +97,9 @@ async fn start_scheduler(
|
||||
ScheduleTaskType::LibraryScan => {
|
||||
run_library_scan(&client, &app_state);
|
||||
}
|
||||
ScheduleTaskType::GeoIpUpdate => {
|
||||
run_geoip_update(&app_state);
|
||||
}
|
||||
}
|
||||
}
|
||||
() = cancel.cancelled() => {
|
||||
@@ -131,7 +156,7 @@ fn run_library_scan(client: &reqwest::Client, app_state: &Arc<AppState>) {
|
||||
LibraryScanTaskOptions {
|
||||
force_rescan: false,
|
||||
message_prefix: "Scheduled ",
|
||||
working_dir: config.working_dir.clone(),
|
||||
storage_dir: config.storage_dir.clone(),
|
||||
},
|
||||
permit,
|
||||
);
|
||||
@@ -140,6 +165,17 @@ fn run_library_scan(client: &reqwest::Client, app_state: &Arc<AppState>) {
|
||||
}
|
||||
}
|
||||
|
||||
fn run_geoip_update(app_state: &Arc<AppState>) {
|
||||
let app_state = Arc::clone(app_state);
|
||||
tokio::spawn(async move {
|
||||
if let Err(err) = update_geoip_db(&app_state).await {
|
||||
if !matches!(err, GeoIpUpdateError::Disabled) {
|
||||
log::error!("Scheduled GeoIp update failed: {err}");
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
pub fn get_process_targets(
|
||||
cfg: &Arc<AppConfig>,
|
||||
process_targets: &Arc<ProcessTargets>,
|
||||
|
||||
+45
-217
@@ -3,8 +3,8 @@ use crate::{
|
||||
auth::generate_password_from_input,
|
||||
model::validate_library_paths_from_dto,
|
||||
utils::{
|
||||
file_exists, get_default_path, get_default_web_root_path, read_api_proxy_file, read_config_file,
|
||||
read_sources_file, read_templates_file, resolve_template_persist_file_path, sanitize_sources_for_persist,
|
||||
file_exists, get_default_web_root_path_for_home, read_api_proxy_file, read_config_file, read_sources_file,
|
||||
read_templates_file, resolve_template_persist_file_path, sanitize_sources_for_persist,
|
||||
},
|
||||
};
|
||||
use axum::{
|
||||
@@ -26,10 +26,10 @@ use shared::{
|
||||
ApiProxyConfigDto, ApiProxyServerInfoDto, AppConfigDto, ConfigApiDto, ConfigDto, ConfigPaths, PatternTemplate,
|
||||
SourcesConfigDto, TemplateDefinitionDto, TokenResponse, WebAuthConfigDto, WebUiConfigDto, TOKEN_NO_AUTH,
|
||||
},
|
||||
utils::{default_kick_secs, hex_encode, DEFAULT_PORT, DEFAULT_WORKING_DIR, USER_FILE},
|
||||
utils::{default_kick_secs, hex_encode, DEFAULT_PORT, DEFAULT_STORAGE_DIR, USER_FILE},
|
||||
};
|
||||
use std::{
|
||||
collections::{HashMap, HashSet},
|
||||
collections::HashSet,
|
||||
io::ErrorKind,
|
||||
net::{SocketAddr, UdpSocket},
|
||||
path::{Component, Path as FsPath, PathBuf},
|
||||
@@ -40,10 +40,9 @@ use std::{
|
||||
};
|
||||
use tokio::sync::{oneshot, Mutex, RwLock};
|
||||
use tower_http::services::ServeDir;
|
||||
use shared::utils::DEFAULT_CUSTOM_STREAM_RESPONSE_PATH;
|
||||
|
||||
const DEFAULT_SETUP_HOST: &str = "0.0.0.0";
|
||||
const DEFAULT_SETUP_CUSTOM_STREAM_RESPONSE_PATH: &str = "./resources";
|
||||
const SETUP_REDACTED_SECRET_VALUE: &str = "__TULIPROX_SETUP_REDACTED__";
|
||||
|
||||
#[derive(Clone, Serialize, Deserialize)]
|
||||
pub struct SetupWebUserCredentialDto {
|
||||
@@ -69,6 +68,7 @@ pub struct SetupCompleteRequestDto {
|
||||
|
||||
struct SetupModeState {
|
||||
draft: Arc<RwLock<AppConfigDto>>,
|
||||
home_path: String,
|
||||
output_dir: PathBuf,
|
||||
config_file_path: PathBuf,
|
||||
source_file_path: PathBuf,
|
||||
@@ -166,7 +166,7 @@ fn create_default_api_proxy_server() -> ApiProxyServerInfoDto {
|
||||
}
|
||||
}
|
||||
|
||||
fn create_default_config_dto() -> ConfigDto {
|
||||
fn create_default_config_dto(home_path: &str) -> ConfigDto {
|
||||
let auth = WebAuthConfigDto {
|
||||
issuer: "tuliprox".to_string(),
|
||||
secret: generate_web_auth_secret(),
|
||||
@@ -180,18 +180,20 @@ fn create_default_config_dto() -> ConfigDto {
|
||||
api: ConfigApiDto {
|
||||
host: DEFAULT_SETUP_HOST.to_string(),
|
||||
port: DEFAULT_PORT,
|
||||
web_root: get_default_web_root_path().display().to_string(),
|
||||
web_root: get_default_web_root_path_for_home(FsPath::new(home_path))
|
||||
.display()
|
||||
.to_string(),
|
||||
},
|
||||
working_dir: get_default_path(DEFAULT_WORKING_DIR).display().to_string(),
|
||||
custom_stream_response_path: Some(DEFAULT_SETUP_CUSTOM_STREAM_RESPONSE_PATH.to_string()),
|
||||
storage_dir: Some(DEFAULT_STORAGE_DIR.to_string()),
|
||||
custom_stream_response_path: Some(DEFAULT_CUSTOM_STREAM_RESPONSE_PATH.to_string()),
|
||||
web_ui: Some(web_ui),
|
||||
..ConfigDto::default()
|
||||
}
|
||||
}
|
||||
|
||||
fn create_default_draft() -> AppConfigDto {
|
||||
fn create_default_draft(home_path: &str) -> AppConfigDto {
|
||||
AppConfigDto {
|
||||
config: create_default_config_dto(),
|
||||
config: create_default_config_dto(home_path),
|
||||
sources: SourcesConfigDto::default(),
|
||||
mappings: None,
|
||||
templates: None,
|
||||
@@ -204,7 +206,7 @@ fn create_default_draft() -> AppConfigDto {
|
||||
}
|
||||
|
||||
async fn build_initial_draft(paths: &ConfigPaths) -> AppConfigDto {
|
||||
let mut draft = create_default_draft();
|
||||
let mut draft = create_default_draft(paths.home_path.as_str());
|
||||
|
||||
if file_exists(&paths.config_file_path) {
|
||||
match read_config_file(paths.config_file_path.as_str(), true, false) {
|
||||
@@ -251,7 +253,7 @@ fn generate_web_auth_secret() -> String {
|
||||
hex_encode(&secret).to_lowercase()
|
||||
}
|
||||
|
||||
fn ensure_setup_defaults(config: &mut ConfigDto) {
|
||||
fn ensure_setup_defaults(config: &mut ConfigDto, home_path: &str) {
|
||||
if config.api.host.trim().is_empty() {
|
||||
config.api.host = DEFAULT_SETUP_HOST.to_string();
|
||||
}
|
||||
@@ -259,13 +261,15 @@ fn ensure_setup_defaults(config: &mut ConfigDto) {
|
||||
config.api.port = DEFAULT_PORT;
|
||||
}
|
||||
if config.api.web_root.trim().is_empty() {
|
||||
config.api.web_root = get_default_web_root_path().display().to_string();
|
||||
config.api.web_root = get_default_web_root_path_for_home(FsPath::new(home_path))
|
||||
.display()
|
||||
.to_string();
|
||||
}
|
||||
if config.working_dir.trim().is_empty() {
|
||||
config.working_dir = get_default_path(DEFAULT_WORKING_DIR).display().to_string();
|
||||
if config.storage_dir.as_ref().is_none_or(|dir| dir.trim().is_empty()) {
|
||||
config.storage_dir = Some(DEFAULT_STORAGE_DIR.to_string());
|
||||
}
|
||||
if config.custom_stream_response_path.as_ref().is_none_or(|path| path.trim().is_empty()) {
|
||||
config.custom_stream_response_path = Some(DEFAULT_SETUP_CUSTOM_STREAM_RESPONSE_PATH.to_string());
|
||||
config.custom_stream_response_path = Some(DEFAULT_CUSTOM_STREAM_RESPONSE_PATH.to_string());
|
||||
}
|
||||
|
||||
if config.web_ui.is_none() {
|
||||
@@ -288,7 +292,7 @@ fn ensure_setup_defaults(config: &mut ConfigDto) {
|
||||
}
|
||||
}
|
||||
|
||||
fn setup_bind_values(draft: &AppConfigDto) -> (String, u16, PathBuf) {
|
||||
fn setup_bind_values(draft: &AppConfigDto, home_path: &str) -> (String, u16, PathBuf) {
|
||||
let host = if draft.config.api.host.trim().is_empty() {
|
||||
DEFAULT_SETUP_HOST.to_string()
|
||||
} else {
|
||||
@@ -296,18 +300,18 @@ fn setup_bind_values(draft: &AppConfigDto) -> (String, u16, PathBuf) {
|
||||
};
|
||||
let port = if draft.config.api.port == 0 { DEFAULT_PORT } else { draft.config.api.port };
|
||||
let web_root = if draft.config.api.web_root.trim().is_empty() {
|
||||
get_default_web_root_path()
|
||||
get_default_web_root_path_for_home(FsPath::new(home_path))
|
||||
} else {
|
||||
PathBuf::from(&draft.config.api.web_root)
|
||||
};
|
||||
(host, port, web_root)
|
||||
}
|
||||
|
||||
fn resolve_setup_web_dir(web_root: &FsPath) -> Option<PathBuf> {
|
||||
fn resolve_setup_web_dir(web_root: &FsPath, home_path: &str) -> Option<PathBuf> {
|
||||
if web_root.exists() && web_root.is_dir() {
|
||||
return Some(web_root.to_path_buf());
|
||||
}
|
||||
let fallback = get_default_web_root_path();
|
||||
let fallback = get_default_web_root_path_for_home(FsPath::new(home_path));
|
||||
if fallback.exists() && fallback.is_dir() {
|
||||
return Some(fallback);
|
||||
}
|
||||
@@ -322,126 +326,6 @@ fn api_proxy_or_default(draft: &AppConfigDto) -> ApiProxyConfigDto {
|
||||
})
|
||||
}
|
||||
|
||||
fn is_setup_redacted_value(value: &str) -> bool { value == SETUP_REDACTED_SECRET_VALUE }
|
||||
|
||||
fn redact_non_empty_secret(value: &mut String) {
|
||||
if !value.trim().is_empty() {
|
||||
*value = SETUP_REDACTED_SECRET_VALUE.to_string();
|
||||
}
|
||||
}
|
||||
|
||||
fn redact_optional_secret(value: &mut Option<String>) {
|
||||
if value.as_ref().is_some_and(|entry| !entry.trim().is_empty()) {
|
||||
*value = Some(SETUP_REDACTED_SECRET_VALUE.to_string());
|
||||
}
|
||||
}
|
||||
|
||||
fn redact_api_proxy_user_credentials(api_proxy: &mut ApiProxyConfigDto) {
|
||||
for target in &mut api_proxy.user {
|
||||
for user in &mut target.credentials {
|
||||
redact_non_empty_secret(&mut user.password);
|
||||
redact_optional_secret(&mut user.token);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn redact_app_config_for_setup(mut app_config: AppConfigDto) -> AppConfigDto {
|
||||
if let Some(web_ui) = app_config.config.web_ui.as_mut() {
|
||||
if let Some(auth) = web_ui.auth.as_mut() {
|
||||
redact_non_empty_secret(&mut auth.secret);
|
||||
}
|
||||
}
|
||||
if let Some(api_proxy) = app_config.api_proxy.as_mut() {
|
||||
redact_api_proxy_user_credentials(api_proxy);
|
||||
}
|
||||
app_config
|
||||
}
|
||||
|
||||
fn redact_api_proxy_for_setup(mut api_proxy: ApiProxyConfigDto) -> ApiProxyConfigDto {
|
||||
redact_api_proxy_user_credentials(&mut api_proxy);
|
||||
api_proxy
|
||||
}
|
||||
|
||||
fn restore_redacted_web_auth_secret(app_config: &mut AppConfigDto, draft: &AppConfigDto) {
|
||||
let Some(web_ui) = app_config.config.web_ui.as_mut() else {
|
||||
return;
|
||||
};
|
||||
let Some(auth) = web_ui.auth.as_mut() else {
|
||||
return;
|
||||
};
|
||||
if !is_setup_redacted_value(&auth.secret) {
|
||||
return;
|
||||
}
|
||||
|
||||
let draft_secret = draft
|
||||
.config
|
||||
.web_ui
|
||||
.as_ref()
|
||||
.and_then(|draft_web_ui| draft_web_ui.auth.as_ref())
|
||||
.map(|draft_auth| draft_auth.secret.trim().to_string())
|
||||
.filter(|secret| !secret.is_empty());
|
||||
if let Some(draft_secret) = draft_secret {
|
||||
auth.secret = draft_secret;
|
||||
}
|
||||
}
|
||||
|
||||
fn restore_redacted_api_proxy_credentials(
|
||||
api_proxy: &mut ApiProxyConfigDto,
|
||||
draft_api_proxy: Option<&ApiProxyConfigDto>,
|
||||
) {
|
||||
let mut credentials_by_username: HashMap<String, (String, Option<String>)> = HashMap::new();
|
||||
if let Some(draft) = draft_api_proxy {
|
||||
for target in &draft.user {
|
||||
for user in &target.credentials {
|
||||
credentials_by_username.insert(user.username.clone(), (user.password.clone(), user.token.clone()));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for target in &mut api_proxy.user {
|
||||
for user in &mut target.credentials {
|
||||
let draft_credentials = credentials_by_username.get(&user.username);
|
||||
if is_setup_redacted_value(&user.password) {
|
||||
if let Some((password, _)) = draft_credentials {
|
||||
user.password = password.clone();
|
||||
}
|
||||
}
|
||||
if user.token.as_deref().is_some_and(is_setup_redacted_value) {
|
||||
if let Some((_, token)) = draft_credentials {
|
||||
user.token = token.clone();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn restore_redacted_setup_values(app_config: &mut AppConfigDto, draft: &AppConfigDto) {
|
||||
restore_redacted_web_auth_secret(app_config, draft);
|
||||
if let Some(api_proxy) = app_config.api_proxy.as_mut() {
|
||||
restore_redacted_api_proxy_credentials(api_proxy, draft.api_proxy.as_ref());
|
||||
}
|
||||
}
|
||||
|
||||
fn has_unresolved_redacted_setup_values(app_config: &AppConfigDto) -> bool {
|
||||
if app_config
|
||||
.config
|
||||
.web_ui
|
||||
.as_ref()
|
||||
.and_then(|web_ui| web_ui.auth.as_ref())
|
||||
.is_some_and(|auth| is_setup_redacted_value(&auth.secret))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
app_config.api_proxy.as_ref().is_some_and(|api_proxy| {
|
||||
api_proxy.user.iter().any(|target| {
|
||||
target.credentials.iter().any(|user| {
|
||||
is_setup_redacted_value(&user.password) || user.token.as_deref().is_some_and(is_setup_redacted_value)
|
||||
})
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
fn setup_templates_from_request(app_config: &AppConfigDto) -> Option<Vec<PatternTemplate>> {
|
||||
app_config
|
||||
.templates
|
||||
@@ -518,12 +402,12 @@ async fn setup_token_refresh() -> impl IntoResponse + Send {
|
||||
|
||||
async fn setup_get_config(State(state): State<Arc<SetupModeState>>) -> impl IntoResponse + Send {
|
||||
let draft = state.draft.read().await.clone();
|
||||
axum::Json(redact_app_config_for_setup(draft)).into_response()
|
||||
axum::Json(draft).into_response()
|
||||
}
|
||||
|
||||
async fn setup_get_api_proxy(State(state): State<Arc<SetupModeState>>) -> impl IntoResponse + Send {
|
||||
let draft = state.draft.read().await.clone();
|
||||
axum::Json(redact_api_proxy_for_setup(api_proxy_or_default(&draft))).into_response()
|
||||
axum::Json(api_proxy_or_default(&draft)).into_response()
|
||||
}
|
||||
|
||||
async fn setup_config_json(State(state): State<Arc<SetupModeState>>) -> impl IntoResponse + Send {
|
||||
@@ -751,19 +635,7 @@ async fn setup_complete_inner(
|
||||
state: Arc<SetupModeState>,
|
||||
mut req: SetupCompleteRequestDto,
|
||||
) -> axum::response::Response {
|
||||
let draft_snapshot = state.draft.read().await.clone();
|
||||
restore_redacted_setup_values(&mut req.app_config, &draft_snapshot);
|
||||
if has_unresolved_redacted_setup_values(&req.app_config) {
|
||||
return (
|
||||
StatusCode::BAD_REQUEST,
|
||||
axum::Json(json!({
|
||||
"error": "One or more redacted secret fields could not be restored. Please re-enter secret values before saving setup."
|
||||
})),
|
||||
)
|
||||
.into_response();
|
||||
}
|
||||
|
||||
ensure_setup_defaults(&mut req.app_config.config);
|
||||
ensure_setup_defaults(&mut req.app_config.config, state.home_path.as_str());
|
||||
|
||||
if let Err(err) = req.app_config.config.prepare(false) {
|
||||
return (StatusCode::BAD_REQUEST, axum::Json(json!({ "error": err.to_string() }))).into_response();
|
||||
@@ -983,13 +855,14 @@ fn create_compression_layer() -> tower_http::compression::CompressionLayer {
|
||||
|
||||
pub async fn start_setup_server(paths: &ConfigPaths, missing_files: &[String]) -> Result<(), TuliproxError> {
|
||||
let draft = build_initial_draft(paths).await;
|
||||
let (host, port, web_root) = setup_bind_values(&draft);
|
||||
let web_dir = resolve_setup_web_dir(&web_root)
|
||||
let (host, port, web_root) = setup_bind_values(&draft, paths.home_path.as_str());
|
||||
let web_dir = resolve_setup_web_dir(&web_root, paths.home_path.as_str())
|
||||
.ok_or_else(|| info_err!("Setup mode requires a web directory. Tried '{}'", web_root.display(),))?;
|
||||
|
||||
let (shutdown_tx, shutdown_rx) = oneshot::channel::<()>();
|
||||
let state = Arc::new(SetupModeState {
|
||||
draft: Arc::new(RwLock::new(draft)),
|
||||
home_path: paths.home_path.clone(),
|
||||
output_dir: PathBuf::from(&paths.config_path),
|
||||
config_file_path: PathBuf::from(&paths.config_file_path),
|
||||
source_file_path: PathBuf::from(&paths.sources_file_path),
|
||||
@@ -1001,7 +874,7 @@ pub async fn start_setup_server(paths: &ConfigPaths, missing_files: &[String]) -
|
||||
shutdown_tx: Mutex::new(Some(shutdown_tx)),
|
||||
});
|
||||
|
||||
info!("Setup mode enabled. Missing required config files: {}", missing_files.join(", "));
|
||||
info!("Setup mode enabled.");
|
||||
info!("Setup output directory: {}", state.output_dir.display());
|
||||
info!("Setup web root: {}", state.web_dir.display());
|
||||
info!("Setup server running: http://{host}:{port}");
|
||||
@@ -1048,25 +921,12 @@ pub async fn start_setup_server(paths: &ConfigPaths, missing_files: &[String]) -
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{
|
||||
has_unresolved_redacted_setup_values, redact_app_config_for_setup, restore_redacted_setup_values,
|
||||
SETUP_REDACTED_SECRET_VALUE,
|
||||
};
|
||||
use shared::model::{ApiProxyConfigDto, AppConfigDto, TargetUserDto, WebAuthConfigDto, WebUiConfigDto};
|
||||
use super::api_proxy_or_default;
|
||||
use shared::model::{ApiProxyConfigDto, AppConfigDto, TargetUserDto};
|
||||
|
||||
fn sample_app_config() -> AppConfigDto {
|
||||
AppConfigDto {
|
||||
config: shared::model::ConfigDto {
|
||||
web_ui: Some(WebUiConfigDto {
|
||||
auth: Some(WebAuthConfigDto {
|
||||
issuer: "tuliprox".to_string(),
|
||||
secret: "very-secret-value".to_string(),
|
||||
..Default::default()
|
||||
}),
|
||||
..Default::default()
|
||||
}),
|
||||
..Default::default()
|
||||
},
|
||||
#[test]
|
||||
fn setup_api_proxy_or_default_preserves_existing_credentials() {
|
||||
let app_config = AppConfigDto {
|
||||
api_proxy: Some(ApiProxyConfigDto {
|
||||
user: vec![TargetUserDto {
|
||||
target: "target-a".to_string(),
|
||||
@@ -1080,49 +940,17 @@ mod tests {
|
||||
..Default::default()
|
||||
}),
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn setup_redaction_masks_web_auth_and_api_proxy_credentials() {
|
||||
let redacted = redact_app_config_for_setup(sample_app_config());
|
||||
|
||||
let auth_secret =
|
||||
redacted.config.web_ui.as_ref().and_then(|web_ui| web_ui.auth.as_ref()).map(|auth| auth.secret.as_str());
|
||||
assert_eq!(auth_secret, Some(SETUP_REDACTED_SECRET_VALUE));
|
||||
|
||||
let creds = &redacted.api_proxy.expect("api_proxy should be present").user[0].credentials[0];
|
||||
assert_eq!(creds.password, SETUP_REDACTED_SECRET_VALUE);
|
||||
assert_eq!(creds.token.as_deref(), Some(SETUP_REDACTED_SECRET_VALUE));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn setup_restore_replaces_redacted_values_from_existing_draft() {
|
||||
let draft = sample_app_config();
|
||||
let mut submitted = redact_app_config_for_setup(draft.clone());
|
||||
|
||||
restore_redacted_setup_values(&mut submitted, &draft);
|
||||
|
||||
let auth_secret =
|
||||
submitted.config.web_ui.as_ref().and_then(|web_ui| web_ui.auth.as_ref()).map(|auth| auth.secret.as_str());
|
||||
assert_eq!(auth_secret, Some("very-secret-value"));
|
||||
|
||||
let creds = &submitted.api_proxy.as_ref().expect("api_proxy should be present").user[0].credentials[0];
|
||||
};
|
||||
let api_proxy = api_proxy_or_default(&app_config);
|
||||
let creds = &api_proxy.user[0].credentials[0];
|
||||
assert_eq!(creds.password, "alice-password");
|
||||
assert_eq!(creds.token.as_deref(), Some("alice-token"));
|
||||
assert!(!has_unresolved_redacted_setup_values(&submitted));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn setup_restore_keeps_unmatched_redacted_credentials_as_unresolved() {
|
||||
let draft = sample_app_config();
|
||||
let mut submitted = redact_app_config_for_setup(draft);
|
||||
if let Some(api_proxy) = submitted.api_proxy.as_mut() {
|
||||
api_proxy.user[0].credentials[0].username = "bob".to_string();
|
||||
}
|
||||
|
||||
restore_redacted_setup_values(&mut submitted, &AppConfigDto::default());
|
||||
|
||||
assert!(has_unresolved_redacted_setup_values(&submitted));
|
||||
fn setup_api_proxy_or_default_creates_server_when_missing() {
|
||||
let api_proxy = api_proxy_or_default(&AppConfigDto::default());
|
||||
assert_eq!(api_proxy.server.len(), 1);
|
||||
assert!(!api_proxy.server[0].host.trim().is_empty());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -31,7 +31,7 @@ pub struct LibraryProcessor {
|
||||
|
||||
pub fn resolve_metadata_storage_path(
|
||||
metadata_update_config: Option<&MetadataUpdateConfig>,
|
||||
working_dir: &str,
|
||||
storage_dir: &str,
|
||||
) -> PathBuf {
|
||||
let configured_path = metadata_update_config.map_or_else(
|
||||
|| PathBuf::from(shared::utils::default_metadata_path()),
|
||||
@@ -46,7 +46,7 @@ pub fn resolve_metadata_storage_path(
|
||||
if configured_path.is_absolute() {
|
||||
configured_path.clean()
|
||||
} else {
|
||||
PathBuf::from(working_dir).join(configured_path).clean()
|
||||
PathBuf::from(storage_dir).join(configured_path).clean()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -61,7 +61,7 @@ impl LibraryProcessor {
|
||||
config
|
||||
.library
|
||||
.as_ref()
|
||||
.map(|lib_cfg| Self::new(lib_cfg.clone(), config.metadata_update.as_ref(), client, &config.working_dir))
|
||||
.map(|lib_cfg| Self::new(lib_cfg.clone(), config.metadata_update.as_ref(), client, &config.storage_dir))
|
||||
}
|
||||
|
||||
// Creates a new Library processor with the given configuration
|
||||
@@ -69,9 +69,9 @@ impl LibraryProcessor {
|
||||
config: LibraryConfig,
|
||||
metadata_update_config: Option<&MetadataUpdateConfig>,
|
||||
client: reqwest::Client,
|
||||
working_dir: &str,
|
||||
storage_dir: &str,
|
||||
) -> Self {
|
||||
let storage_path = resolve_metadata_storage_path(metadata_update_config, working_dir);
|
||||
let storage_path = resolve_metadata_storage_path(metadata_update_config, storage_dir);
|
||||
let scanner = LibraryScanner::new(config.clone());
|
||||
let storage = MetadataStorage::new(storage_path);
|
||||
let resolver = MetadataResolver::from_config(Some(&config), metadata_update_config, client, Some(storage.clone()));
|
||||
|
||||
@@ -473,7 +473,7 @@ impl TmdbSeason {
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{resolve_tmdb_image_url, TmdbMovieDetails, TMDB_IMAGE_BASE_URL};
|
||||
use super::{resolve_tmdb_image_url, TMDB_IMAGE_BASE_URL};
|
||||
|
||||
#[test]
|
||||
fn test_resolve_tmdb_url_with_absolute_http() {
|
||||
|
||||
+52
-20
@@ -30,7 +30,7 @@ use clap::Parser;
|
||||
use log::{error, info, warn};
|
||||
use shared::{
|
||||
model::ConfigPaths,
|
||||
utils::{CONFIG_FILE, SOURCE_FILE},
|
||||
utils::{CONFIG_FILE, CONFIG_PATH, DEFAULT_STORAGE_DIR, SOURCE_FILE},
|
||||
};
|
||||
use std::{
|
||||
fs::File,
|
||||
@@ -45,6 +45,10 @@ use std::{
|
||||
#[command(version)]
|
||||
#[command(about = "Extended playlist proxy", long_about = None)]
|
||||
struct Args {
|
||||
/// The home directory (base for config, storage, backup, downloads)
|
||||
#[arg(short = 'H', long = "home")]
|
||||
home: Option<String>,
|
||||
|
||||
/// The config directory
|
||||
#[arg(short = 'p', long = "config-path")]
|
||||
config_path: Option<String>,
|
||||
@@ -217,21 +221,13 @@ fn run_startup_bplustree_migration(config_paths: &ConfigPaths) {
|
||||
return;
|
||||
}
|
||||
|
||||
let config_dto = match utils::read_config_file(config_paths.config_file_path.as_str(), true, false) {
|
||||
Ok(cfg) => cfg,
|
||||
Err(err) => {
|
||||
warn!("Skipping B+Tree startup migration because config could not be read: {err}");
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let mut roots: Vec<PathBuf> = vec![PathBuf::from(&config_paths.config_path)];
|
||||
let working_dir = PathBuf::from(config_dto.working_dir);
|
||||
if !roots.iter().any(|root| root == &working_dir) {
|
||||
roots.push(working_dir.clone());
|
||||
let storage_dir = PathBuf::from(&config_paths.storage_path);
|
||||
if !roots.iter().any(|root| root == &storage_dir) {
|
||||
roots.push(storage_dir.clone());
|
||||
}
|
||||
|
||||
match migrate_bplustree_databases_with_marker(&roots, &working_dir) {
|
||||
match migrate_bplustree_databases_with_marker(&roots, &storage_dir) {
|
||||
Ok(stats) => {
|
||||
if stats.skipped_by_marker {
|
||||
info!("B+Tree startup migration skipped (marker already present)");
|
||||
@@ -260,7 +256,7 @@ fn print_info(app_config: &AppConfig) {
|
||||
let paths = <Arc<ArcSwap<ConfigPaths>> as Access<ConfigPaths>>::load(&app_config.paths);
|
||||
info!("Current time: {}", chrono::offset::Local::now().format("%Y-%m-%d %H:%M:%S"));
|
||||
info!("Temp dir: {}", tempfile::env::temp_dir().display());
|
||||
info!("Working dir: {:?}", &config.working_dir);
|
||||
info!("Storage dir: {:?}", &config.storage_dir);
|
||||
info!("Config dir: {:?}", &paths.config_path);
|
||||
info!("Config file: {:?}", &paths.config_file_path);
|
||||
info!("Source file: {:?}", &paths.sources_file_path);
|
||||
@@ -281,7 +277,7 @@ fn print_info(app_config: &AppConfig) {
|
||||
|
||||
if let Some(cache) = config.reverse_proxy.as_ref().and_then(|r| r.cache.as_ref()) {
|
||||
if cache.enabled {
|
||||
info!("Cache dir: {}", cache.dir);
|
||||
info!("Cache dir: {}", cache.directory);
|
||||
}
|
||||
}
|
||||
if let Some(metadata_update) = config.metadata_update.as_ref() {
|
||||
@@ -293,11 +289,31 @@ fn print_info(app_config: &AppConfig) {
|
||||
}
|
||||
|
||||
fn get_file_paths(args: &Args) -> ConfigPaths {
|
||||
let raw_path = args
|
||||
.config_path
|
||||
let resolve_storage_path = |home_path: &str, storage_dir: Option<&str>| {
|
||||
let path = storage_dir.map(str::trim).filter(|value| !value.is_empty()).map(PathBuf::from).map_or_else(
|
||||
|| utils::get_default_path_for_home(Path::new(home_path), DEFAULT_STORAGE_DIR),
|
||||
|configured_path| {
|
||||
if configured_path.is_relative() {
|
||||
PathBuf::from(home_path).join(configured_path)
|
||||
} else {
|
||||
configured_path
|
||||
}
|
||||
},
|
||||
);
|
||||
utils::resolve_directory_path(path.to_string_lossy().as_ref())
|
||||
};
|
||||
|
||||
let home_path = args
|
||||
.home
|
||||
.as_ref()
|
||||
.filter(|p| !p.trim().is_empty())
|
||||
.map_or_else(utils::get_default_config_path, |p| resolve_env_var(p));
|
||||
.map_or_else(utils::get_home_path, |p| PathBuf::from(resolve_env_var(p)));
|
||||
let home_path = utils::resolve_directory_path(home_path.to_string_lossy().as_ref());
|
||||
|
||||
let raw_path = args.config_path.as_ref().filter(|p| !p.trim().is_empty()).map_or_else(
|
||||
|| utils::get_default_path_for_home(Path::new(&home_path), CONFIG_PATH).to_string_lossy().to_string(),
|
||||
|p| resolve_env_var(p),
|
||||
);
|
||||
let config_path: String = utils::resolve_directory_path(&raw_path);
|
||||
|
||||
let config_file: String = resolve_env_var(
|
||||
@@ -318,11 +334,27 @@ fn get_file_paths(args: &Args) -> ConfigPaths {
|
||||
.as_ref()
|
||||
.map_or_else(|| utils::get_default_sources_file_path(&config_path), ToString::to_string),
|
||||
);
|
||||
let mappings_file = args.mapping_file.as_ref().map(|p| resolve_env_var(p));
|
||||
let template_file = args.template_file.as_ref().map(|p| resolve_env_var(p));
|
||||
let mappings_file = args.mapping_file.as_ref().map(|p| {
|
||||
let path = resolve_env_var(p);
|
||||
utils::resolve_mapping_file_path(config_path.as_str(), Some(path.as_str()))
|
||||
});
|
||||
let template_file = args.template_file.as_ref().map(|p| {
|
||||
let path = resolve_env_var(p);
|
||||
utils::resolve_template_file_path(config_path.as_str(), Some(path.as_str()))
|
||||
});
|
||||
let storage_path = if Path::new(&config_file).exists() {
|
||||
match utils::read_config_file(&config_file, true, false) {
|
||||
Ok(cfg) => resolve_storage_path(&home_path, cfg.storage_dir.as_deref()),
|
||||
Err(err) => exit!("Can't read config file {} while resolving storage path: {err}", config_file),
|
||||
}
|
||||
} else {
|
||||
resolve_storage_path(&home_path, None)
|
||||
};
|
||||
|
||||
ConfigPaths {
|
||||
home_path,
|
||||
config_path,
|
||||
storage_path,
|
||||
config_file_path: config_file,
|
||||
sources_file_path: sources_file,
|
||||
mapping_file_path: mappings_file, // need to be set after config read
|
||||
|
||||
@@ -309,7 +309,9 @@ mod tests {
|
||||
api_proxy: Arc::new(ArcSwapOption::default()),
|
||||
file_locks: Arc::new(FileLockManager::default()),
|
||||
paths: Arc::new(ArcSwap::from_pointee(ConfigPaths {
|
||||
home_path: String::new(),
|
||||
config_path: String::new(),
|
||||
storage_path: String::new(),
|
||||
config_file_path: String::new(),
|
||||
sources_file_path: String::new(),
|
||||
mapping_file_path: None,
|
||||
|
||||
@@ -329,9 +329,10 @@ impl AppConfig {
|
||||
fn set_mapping_path(&self, mapping_path: Option<&str>) {
|
||||
let paths_guard = self.paths.load();
|
||||
let old_path = paths_guard.mapping_file_path.as_deref();
|
||||
let new_path = mapping_path
|
||||
.map(ToString::to_string)
|
||||
.or_else(|| Some(utils::get_default_mappings_path(&paths_guard.config_path)));
|
||||
let new_path = Some(utils::resolve_mapping_file_path(
|
||||
paths_guard.config_path.as_str(),
|
||||
mapping_path,
|
||||
));
|
||||
|
||||
if old_path != new_path.as_deref() {
|
||||
let mut new_paths = (**paths_guard).clone();
|
||||
@@ -343,9 +344,10 @@ impl AppConfig {
|
||||
fn set_template_path(&self, template_path: Option<&str>) {
|
||||
let paths_guard = self.paths.load();
|
||||
let old_path = paths_guard.template_file_path.as_deref();
|
||||
let new_path = template_path
|
||||
.map(ToString::to_string)
|
||||
.or_else(|| Some(utils::get_default_templates_path(&paths_guard.config_path)));
|
||||
let new_path = Some(utils::resolve_template_file_path(
|
||||
paths_guard.config_path.as_str(),
|
||||
template_path,
|
||||
));
|
||||
|
||||
if old_path != new_path.as_deref() {
|
||||
let mut new_paths = (**paths_guard).clone();
|
||||
@@ -406,7 +408,7 @@ impl AppConfig {
|
||||
}
|
||||
|
||||
let path = PathBuf::from(custom_stream_response_path);
|
||||
let path = utils::make_path_absolute(&path, &config.working_dir);
|
||||
let path = utils::make_path_absolute(&path, &config.storage_dir);
|
||||
|
||||
let paths = self.paths.load_full();
|
||||
let mut new_paths = paths.as_ref().clone();
|
||||
|
||||
@@ -8,21 +8,19 @@ use log::{error, info};
|
||||
use path_clean::PathClean;
|
||||
use shared::error::TuliproxError;
|
||||
use shared::model::{ConfigDto, HdHomeRunDeviceOverview};
|
||||
use shared::utils::{default_grace_period_millis, default_grace_period_timeout_secs, set_sanitize_sensitive_info, DEFAULT_WEB_DIR};
|
||||
use shared::utils::{default_grace_period_millis, default_grace_period_timeout_secs, set_sanitize_sensitive_info, DEFAULT_BACKUP_DIR, DEFAULT_CACHE_DIR, DEFAULT_DOWNLOAD_DIR, DEFAULT_STORAGE_DIR, DEFAULT_STORAGE_TEMP_DIR, DEFAULT_USER_CONFIG_DIR};
|
||||
use std::borrow::Cow;
|
||||
use std::path::{Path, PathBuf};
|
||||
use crate::utils::get_default_path;
|
||||
|
||||
const DEFAULT_BACKUP_DIR: &str = "backup";
|
||||
use crate::utils::get_default_path_for_home;
|
||||
|
||||
fn create_directories(cfg: &Config, temp_path: &Path) {
|
||||
// Collect the paths into a vector.
|
||||
let paths_strings = [
|
||||
Some(cfg.working_dir.clone()),
|
||||
Some(cfg.storage_dir.clone()),
|
||||
cfg.backup_dir.clone(),
|
||||
cfg.user_config_dir.clone(),
|
||||
cfg.video.as_ref().and_then(|v| v.download.as_ref()).map(|d| d.directory.clone()),
|
||||
cfg.reverse_proxy.as_ref().and_then(|r| r.cache.as_ref().and_then(|c| if c.enabled { Some(c.dir.clone()) } else { None })),
|
||||
cfg.reverse_proxy.as_ref().and_then(|r| r.cache.as_ref().and_then(|c| if c.enabled { Some(c.directory.clone()) } else { None })),
|
||||
cfg.metadata_update.as_ref().map(|m| m.cache_path.clone()),
|
||||
];
|
||||
|
||||
@@ -67,7 +65,7 @@ impl Default for GracePeriodOptions {
|
||||
pub struct Config {
|
||||
pub process_parallel: bool,
|
||||
pub api: ConfigApi,
|
||||
pub working_dir: String,
|
||||
pub storage_dir: String,
|
||||
pub default_user_agent: Option<String>,
|
||||
pub backup_dir: Option<String>,
|
||||
pub user_config_dir: Option<String>,
|
||||
@@ -95,24 +93,58 @@ pub struct Config {
|
||||
}
|
||||
|
||||
impl Config {
|
||||
pub fn prepare(&mut self, config_path: &str) -> Result<(), TuliproxError> {
|
||||
let work_dir = &self.working_dir;
|
||||
self.working_dir = utils::resolve_directory_path(work_dir);
|
||||
fn lexical_normalize(path: &Path) -> PathBuf {
|
||||
path.components().collect::<PathBuf>()
|
||||
}
|
||||
|
||||
self.prepare_directories();
|
||||
self.prepare_api_web_root();
|
||||
fn normalize_storage_path(raw_storage_dir: &str, storage_dir: &str, input: &str, default_dir: &str) -> Option<PathBuf> {
|
||||
let trimmed_input = input.trim();
|
||||
if trimmed_input.is_empty() {
|
||||
return Some(PathBuf::from(storage_dir).join(default_dir));
|
||||
}
|
||||
|
||||
let input_path = PathBuf::from(trimmed_input);
|
||||
if input_path.is_absolute() {
|
||||
return None;
|
||||
}
|
||||
|
||||
let mut normalized_relative = Self::lexical_normalize(&input_path);
|
||||
let trimmed_raw_storage = raw_storage_dir.trim();
|
||||
if !trimmed_raw_storage.is_empty() {
|
||||
let raw_path = PathBuf::from(trimmed_raw_storage);
|
||||
if raw_path.is_relative() {
|
||||
let normalized_raw = Self::lexical_normalize(&raw_path);
|
||||
if normalized_raw.components().next().is_some() {
|
||||
while normalized_relative.starts_with(&normalized_raw) {
|
||||
match normalized_relative.strip_prefix(&normalized_raw) {
|
||||
Ok(stripped) => {
|
||||
normalized_relative = Self::lexical_normalize(stripped);
|
||||
}
|
||||
Err(_) => break,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Some(PathBuf::from(storage_dir).join(normalized_relative))
|
||||
}
|
||||
|
||||
pub fn prepare(&mut self, config_path: &str, home_path: &str) -> Result<(), TuliproxError> {
|
||||
|
||||
self.prepare_directories(home_path);
|
||||
if let Some(ref mut webui) = &mut self.web_ui {
|
||||
webui.prepare(config_path)?;
|
||||
}
|
||||
|
||||
if let Some(library) = self.library.as_mut() {
|
||||
library.prepare(&self.working_dir)?;
|
||||
library.prepare(&self.storage_dir)?;
|
||||
}
|
||||
|
||||
if let Some(metadata_update) = self.metadata_update.as_mut() {
|
||||
let meta_path = PathBuf::from(&metadata_update.cache_path);
|
||||
let meta_path = if meta_path.is_relative() {
|
||||
PathBuf::from(&self.working_dir).join(meta_path)
|
||||
PathBuf::from(&self.storage_dir).join(meta_path)
|
||||
} else {
|
||||
meta_path
|
||||
};
|
||||
@@ -125,38 +157,87 @@ impl Config {
|
||||
|
||||
if let Some(video) = self.video.as_mut() {
|
||||
video.prepare();
|
||||
if let Some(download) = video.download.as_mut() {
|
||||
let download_path = PathBuf::from(&download.directory);
|
||||
if download.directory.trim().is_empty() {
|
||||
download.directory = get_default_path_for_home(Path::new(home_path), DEFAULT_DOWNLOAD_DIR)
|
||||
.clean()
|
||||
.to_string_lossy()
|
||||
.to_string();
|
||||
} else if download_path.is_relative() {
|
||||
download.directory = PathBuf::from(home_path)
|
||||
.join(download_path)
|
||||
.clean()
|
||||
.to_string_lossy()
|
||||
.to_string();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn prepare_directories(&mut self) {
|
||||
fn set_directory(path: &mut Option<String>, default_subdir: &str, working_dir: &str) {
|
||||
*path = Some(match path.as_ref() {
|
||||
Some(existing) => existing.to_owned(),
|
||||
None => PathBuf::from(working_dir).join(default_subdir).clean().to_string_lossy().to_string(),
|
||||
});
|
||||
fn prepare_reverse_proxy_cache_dir(&mut self, raw_storage_dir: &str) {
|
||||
let Some(reverse_proxy) = self.reverse_proxy.as_mut() else {
|
||||
return;
|
||||
};
|
||||
let Some(cache) = reverse_proxy.cache.as_mut() else {
|
||||
return;
|
||||
};
|
||||
if !cache.enabled {
|
||||
return;
|
||||
}
|
||||
|
||||
set_directory(&mut self.backup_dir, DEFAULT_BACKUP_DIR, &self.working_dir);
|
||||
set_directory(&mut self.user_config_dir, "user_config", &self.working_dir);
|
||||
let normalized = Self::normalize_storage_path(raw_storage_dir, &self.storage_dir, &cache.directory, DEFAULT_CACHE_DIR)
|
||||
.unwrap_or_else(|| PathBuf::from(cache.directory.trim()));
|
||||
cache.directory = normalized.clean().to_string_lossy().to_string();
|
||||
}
|
||||
|
||||
fn prepare_directories(&mut self, home_path: &str) {
|
||||
let raw_storage_dir = self.storage_dir.trim().to_string();
|
||||
let storage_dir_path = if raw_storage_dir.is_empty() {
|
||||
get_default_path_for_home(Path::new(home_path), DEFAULT_STORAGE_DIR)
|
||||
} else {
|
||||
let configured_storage_path = PathBuf::from(&raw_storage_dir);
|
||||
if configured_storage_path.is_relative() {
|
||||
PathBuf::from(home_path).join(configured_storage_path)
|
||||
} else {
|
||||
configured_storage_path
|
||||
}
|
||||
};
|
||||
self.storage_dir = utils::resolve_directory_path(storage_dir_path.to_string_lossy().as_ref());
|
||||
self.prepare_reverse_proxy_cache_dir(&raw_storage_dir);
|
||||
|
||||
let storage_dir = self.storage_dir.clone();
|
||||
let normalize_optional_path = |value: Option<&str>, default_dir: &str| -> String {
|
||||
let configured = value.unwrap_or_default();
|
||||
let normalized = Self::normalize_storage_path(&raw_storage_dir, &storage_dir, configured, default_dir)
|
||||
.unwrap_or_else(|| PathBuf::from(configured.trim()));
|
||||
normalized.clean().to_string_lossy().to_string()
|
||||
};
|
||||
|
||||
self.backup_dir = Some(normalize_optional_path(self.backup_dir.as_deref(), DEFAULT_BACKUP_DIR));
|
||||
self.user_config_dir = Some(normalize_optional_path(self.user_config_dir.as_deref(), DEFAULT_USER_CONFIG_DIR));
|
||||
self.prepare_api_web_root(home_path);
|
||||
}
|
||||
|
||||
pub fn get_backup_dir(&self) -> Cow<'_, str> {
|
||||
self.backup_dir.as_ref().map_or_else(|| Cow::Borrowed(DEFAULT_BACKUP_DIR), |v| Cow::Borrowed(v))
|
||||
}
|
||||
|
||||
fn prepare_api_web_root(&mut self) {
|
||||
fn prepare_api_web_root(&mut self, home_path: &str) {
|
||||
if self.api.web_root.is_empty() {
|
||||
self.api.web_root = get_default_path(DEFAULT_WEB_DIR).display().to_string();
|
||||
self.api.web_root = utils::get_default_web_root_path_for_home(Path::new(home_path))
|
||||
.display()
|
||||
.to_string();
|
||||
} else {
|
||||
self.api.web_root = utils::make_absolute_path(&self.api.web_root, &self.working_dir);
|
||||
self.api.web_root = utils::make_absolute_path(&self.api.web_root, &self.storage_dir);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn update_runtime(&self) {
|
||||
set_sanitize_sensitive_info(self.log.as_ref().is_none_or(|l| l.sanitize_sensitive_info));
|
||||
let temp_path = PathBuf::from(&self.working_dir).join("tmp");
|
||||
let temp_path = PathBuf::from(&self.storage_dir).join(DEFAULT_STORAGE_TEMP_DIR);
|
||||
create_directories(self, &temp_path);
|
||||
let _ = tempfile::env::override_temp_dir(&temp_path);
|
||||
}
|
||||
@@ -200,7 +281,7 @@ impl From<&ConfigDto> for Config {
|
||||
process_parallel: dto.process_parallel,
|
||||
disk_based_processing: dto.disk_based_processing,
|
||||
api: ConfigApi::from(&dto.api),
|
||||
working_dir: dto.working_dir.clone(),
|
||||
storage_dir: dto.storage_dir.clone().unwrap_or_default(),
|
||||
default_user_agent: dto.default_user_agent.clone(),
|
||||
backup_dir: dto.backup_dir.clone(),
|
||||
user_config_dir: dto.user_config_dir.clone(),
|
||||
|
||||
@@ -5,7 +5,7 @@ use crate::model::macros;
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CacheConfig {
|
||||
pub enabled: bool,
|
||||
pub dir: String,
|
||||
pub directory: String,
|
||||
pub size: usize,
|
||||
pub size_str: Option<String>,
|
||||
}
|
||||
@@ -16,7 +16,7 @@ impl From<&CacheConfigDto> for CacheConfig {
|
||||
Self {
|
||||
enabled: dto.enabled,
|
||||
// Dto prepare should have set the right path
|
||||
dir: dto.dir.as_ref().map_or_else(Default::default, std::string::ToString::to_string),
|
||||
directory: dto.directory.as_ref().map_or_else(Default::default, std::string::ToString::to_string),
|
||||
size_str: dto.size.clone(),
|
||||
size: get_size(dto)
|
||||
}
|
||||
@@ -28,7 +28,7 @@ impl From<&CacheConfig> for CacheConfigDto {
|
||||
Self {
|
||||
enabled: instance.enabled,
|
||||
// Dto prepare should have set the right path
|
||||
dir: Some(instance.dir.clone()),
|
||||
directory: Some(instance.directory.clone()),
|
||||
size: instance.size_str.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -242,6 +242,14 @@ pub struct ConfigInput {
|
||||
}
|
||||
|
||||
impl ConfigInput {
|
||||
#[inline]
|
||||
pub fn get_download_input_type(&self) -> InputType {
|
||||
self.staged
|
||||
.as_ref()
|
||||
.filter(|staged| staged.enabled)
|
||||
.map_or(self.input_type, |staged| staged.input_type)
|
||||
}
|
||||
|
||||
#[inline]
|
||||
pub fn has_flag(&self, flag: ConfigInputFlags) -> bool {
|
||||
self.has_flag_or(flag, false)
|
||||
@@ -734,4 +742,34 @@ mod tests {
|
||||
let err = input.prepare(&[]).unwrap_err();
|
||||
assert!(err.to_string().contains("Malformed provider URL"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_get_download_input_type_uses_staged_when_enabled() {
|
||||
let input = ConfigInput {
|
||||
input_type: InputType::Xtream,
|
||||
staged: Some(StagedInput {
|
||||
enabled: true,
|
||||
input_type: InputType::M3u,
|
||||
..Default::default()
|
||||
}),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
assert_eq!(input.get_download_input_type(), InputType::M3u);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_get_download_input_type_uses_primary_when_staged_disabled() {
|
||||
let input = ConfigInput {
|
||||
input_type: InputType::Xtream,
|
||||
staged: Some(StagedInput {
|
||||
enabled: false,
|
||||
input_type: InputType::M3u,
|
||||
..Default::default()
|
||||
}),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
assert_eq!(input.get_download_input_type(), InputType::Xtream);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
use crate::model::macros;
|
||||
use shared::error::{info_err_res, TuliproxError};
|
||||
use shared::model::{ConfigDto, LibraryConfigDto, LibraryContentType, LibraryMetadataFormat};
|
||||
use shared::utils::DEFAULT_STORAGE_DIR;
|
||||
use shared::utils::Internable;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
@@ -43,14 +44,14 @@ pub struct LibraryConfig {
|
||||
}
|
||||
|
||||
impl LibraryConfig {
|
||||
fn canonicalize_scan_directory_path(path: &str, working_dir: &str) -> Result<String, TuliproxError> {
|
||||
fn canonicalize_scan_directory_path(path: &str, storage_dir: &str) -> Result<String, TuliproxError> {
|
||||
let path = path.trim();
|
||||
if path.is_empty() {
|
||||
return info_err_res!("Library scan directory path cannot be empty");
|
||||
}
|
||||
|
||||
let scan_path = PathBuf::from(path);
|
||||
let scan_path = if scan_path.is_relative() { PathBuf::from(working_dir).join(scan_path) } else { scan_path };
|
||||
let scan_path = if scan_path.is_relative() { PathBuf::from(storage_dir).join(scan_path) } else { scan_path };
|
||||
|
||||
match scan_path.canonicalize() {
|
||||
Ok(path_buf) => Ok(path_buf.to_string_lossy().to_string()),
|
||||
@@ -58,16 +59,16 @@ impl LibraryConfig {
|
||||
}
|
||||
}
|
||||
|
||||
fn canonicalize_scan_directories(&mut self, working_dir: &str) -> Result<(), TuliproxError> {
|
||||
fn canonicalize_scan_directories(&mut self, storage_dir: &str) -> Result<(), TuliproxError> {
|
||||
for dir in &mut self.scan_directories {
|
||||
dir.path = Self::canonicalize_scan_directory_path(&dir.path, working_dir)?;
|
||||
dir.path = Self::canonicalize_scan_directory_path(&dir.path, storage_dir)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn prepare(&mut self, working_dir: &str) -> Result<(), TuliproxError> {
|
||||
pub fn prepare(&mut self, storage_dir: &str) -> Result<(), TuliproxError> {
|
||||
if self.enabled {
|
||||
self.canonicalize_scan_directories(working_dir)?;
|
||||
self.canonicalize_scan_directories(storage_dir)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -119,11 +120,18 @@ pub fn validate_library_paths_from_dto(cfg: &ConfigDto) -> Result<(), TuliproxEr
|
||||
};
|
||||
|
||||
let mut library_cfg = LibraryConfig::from(library_dto);
|
||||
let storage_dir = cfg
|
||||
.storage_dir
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|dir| !dir.is_empty())
|
||||
.unwrap_or(DEFAULT_STORAGE_DIR);
|
||||
// Always validate configured scan directories, even when library is disabled.
|
||||
// This prevents persisting invalid paths that would later break startup when enabled.
|
||||
library_cfg.canonicalize_scan_directories(&cfg.working_dir)?;
|
||||
if library_cfg.enabled {
|
||||
library_cfg.prepare(&cfg.working_dir)?;
|
||||
library_cfg.prepare(storage_dir)?;
|
||||
} else {
|
||||
library_cfg.canonicalize_scan_directories(storage_dir)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -143,10 +151,12 @@ mod tests {
|
||||
#[test]
|
||||
fn validate_library_paths_rejects_missing_scan_directory() {
|
||||
let cfg = ConfigDto {
|
||||
working_dir: std::env::current_dir()
|
||||
.expect("current_dir should be available")
|
||||
.to_string_lossy()
|
||||
.to_string(),
|
||||
storage_dir: Some(
|
||||
std::env::current_dir()
|
||||
.expect("current_dir should be available")
|
||||
.to_string_lossy()
|
||||
.to_string(),
|
||||
),
|
||||
library: Some(LibraryConfigDto {
|
||||
enabled: true,
|
||||
scan_directories: vec![LibraryScanDirectoryDto { path: unique_missing_path(), ..Default::default() }],
|
||||
@@ -162,10 +172,12 @@ mod tests {
|
||||
#[test]
|
||||
fn validate_library_paths_rejects_missing_scan_directory_even_when_disabled() {
|
||||
let cfg = ConfigDto {
|
||||
working_dir: std::env::current_dir()
|
||||
.expect("current_dir should be available")
|
||||
.to_string_lossy()
|
||||
.to_string(),
|
||||
storage_dir: Some(
|
||||
std::env::current_dir()
|
||||
.expect("current_dir should be available")
|
||||
.to_string_lossy()
|
||||
.to_string(),
|
||||
),
|
||||
library: Some(LibraryConfigDto {
|
||||
enabled: false,
|
||||
scan_directories: vec![LibraryScanDirectoryDto { path: unique_missing_path(), ..Default::default() }],
|
||||
@@ -182,10 +194,12 @@ mod tests {
|
||||
#[test]
|
||||
fn validate_library_paths_accepts_existing_scan_directory() {
|
||||
let cfg = ConfigDto {
|
||||
working_dir: std::env::current_dir()
|
||||
.expect("current_dir should be available")
|
||||
.to_string_lossy()
|
||||
.to_string(),
|
||||
storage_dir: Some(
|
||||
std::env::current_dir()
|
||||
.expect("current_dir should be available")
|
||||
.to_string_lossy()
|
||||
.to_string(),
|
||||
),
|
||||
library: Some(LibraryConfigDto {
|
||||
enabled: true,
|
||||
scan_directories: vec![LibraryScanDirectoryDto {
|
||||
|
||||
@@ -182,13 +182,6 @@ impl From<&ConfigProviderDto> for ConfigProvider {
|
||||
fn from(dto: &ConfigProviderDto) -> Self {
|
||||
let dns_cfg = dto.dns.as_ref().map(ProviderDnsConfig::from);
|
||||
let dns_cache = Arc::new(ProviderDnsCache::default());
|
||||
if let Some(dns_dto) = dto.dns.as_ref() {
|
||||
if let Some(resolved) = dns_dto.resolved.as_ref() {
|
||||
for (host, ips) in resolved {
|
||||
dns_cache.store_resolved(host, ips.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
Self {
|
||||
name: dto.name.clone(),
|
||||
urls: dto.urls.clone(),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
use crate::model::macros;
|
||||
use regex::Regex;
|
||||
use shared::model::{VideoConfigDto, VideoDownloadConfigDto};
|
||||
use shared::utils::DEFAULT_DOWNLOAD_DIR;
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
@@ -17,7 +18,7 @@ impl From<&VideoDownloadConfigDto> for VideoDownloadConfig {
|
||||
fn from(dto: &VideoDownloadConfigDto) -> Self {
|
||||
Self {
|
||||
headers: dto.headers.clone(),
|
||||
directory: dto.directory.as_ref().map_or_else(|| "downloads".to_string(), ToString::to_string),
|
||||
directory: dto.directory.as_ref().map_or_else(|| DEFAULT_DOWNLOAD_DIR.to_string(), ToString::to_string),
|
||||
organize_into_directories: dto.organize_into_directories,
|
||||
episode_pattern: dto.episode_pattern.as_ref().and_then(|s| shared::model::REGEX_CACHE.get_or_compile(s)
|
||||
.map_err(|e| log::warn!("Invalid episode_pattern regex '{s}': {e}"))
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
use crate::{
|
||||
api::model::AppState,
|
||||
model::InputSource,
|
||||
repository::get_geoip_path,
|
||||
utils::{request::download_text_content, GeoIp},
|
||||
};
|
||||
use log::{error, info};
|
||||
use shared::{
|
||||
model::{default_geoip_url, InputFetchMethod},
|
||||
utils::Internable,
|
||||
};
|
||||
use std::{collections::HashMap, io::Cursor, sync::Arc};
|
||||
|
||||
#[derive(Debug)]
|
||||
pub(crate) enum GeoIpUpdateError {
|
||||
Disabled,
|
||||
DownloadFailed(String),
|
||||
ProcessFailed(String),
|
||||
UnknownProcessing,
|
||||
}
|
||||
|
||||
impl std::fmt::Display for GeoIpUpdateError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Self::Disabled => write!(f, "GeoIp update is disabled"),
|
||||
Self::DownloadFailed(err) => write!(f, "Failed to download geoip db: {err}"),
|
||||
Self::ProcessFailed(err) => write!(f, "Failed to process geoip db: {err}"),
|
||||
Self::UnknownProcessing => write!(f, "Unknown GeoIp processing error"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for GeoIpUpdateError {}
|
||||
|
||||
pub(crate) async fn update_geoip_db(app_state: &Arc<AppState>) -> Result<(), GeoIpUpdateError> {
|
||||
let config = app_state.app_config.config.load();
|
||||
if let Some(geoip) = config.reverse_proxy.as_ref().and_then(|r| r.geoip.as_ref()) {
|
||||
if geoip.enabled {
|
||||
let geoip_db_path = &*get_geoip_path(&config.storage_dir);
|
||||
let _file_lock = app_state.app_config.file_locks.write_lock(geoip_db_path).await;
|
||||
|
||||
let url = if geoip.url.trim().is_empty() { default_geoip_url() } else { geoip.url.clone() };
|
||||
let input_source = InputSource {
|
||||
name: "GeoIP".intern(),
|
||||
url,
|
||||
provider: None,
|
||||
username: None,
|
||||
password: None,
|
||||
method: InputFetchMethod::GET,
|
||||
headers: HashMap::default(),
|
||||
};
|
||||
return match download_text_content(
|
||||
&app_state.app_config,
|
||||
&app_state.http_client.load(),
|
||||
&input_source,
|
||||
None,
|
||||
None,
|
||||
false,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok((content, _)) => {
|
||||
let reader = Cursor::new(content);
|
||||
let mut geoip = GeoIp::new();
|
||||
let result = {
|
||||
match geoip.import_ipv4_from_csv(reader, geoip_db_path) {
|
||||
Ok(size) => (Some(size), None),
|
||||
Err(err) => (None, Some(err)),
|
||||
}
|
||||
};
|
||||
|
||||
return match result {
|
||||
(Some(_), None) => {
|
||||
info!("GeoIp db updated");
|
||||
app_state.geoip.store(Some(Arc::new(geoip)));
|
||||
Ok(())
|
||||
}
|
||||
(None, Some(err)) => {
|
||||
let error = GeoIpUpdateError::ProcessFailed(err.to_string());
|
||||
error!("{error}");
|
||||
Err(error)
|
||||
}
|
||||
_ => Err(GeoIpUpdateError::UnknownProcessing),
|
||||
};
|
||||
}
|
||||
Err(err) => {
|
||||
let error = GeoIpUpdateError::DownloadFailed(err.to_string());
|
||||
error!("{error}");
|
||||
Err(error)
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
Err(GeoIpUpdateError::Disabled)
|
||||
}
|
||||
@@ -27,9 +27,9 @@ pub struct InputStatus {
|
||||
pub clusters: HashMap<String, ClusterStatus>,
|
||||
}
|
||||
|
||||
pub async fn resolve_input_storage_path(working_dir: &str, input_name: &str) -> PathBuf {
|
||||
if let Ok(path) = get_input_storage_path(input_name, working_dir).await { path } else {
|
||||
build_input_storage_path(input_name, working_dir)
|
||||
pub async fn resolve_input_storage_path(storage_dir: &str, input_name: &str) -> PathBuf {
|
||||
if let Ok(path) = get_input_storage_path(input_name, storage_dir).await { path } else {
|
||||
build_input_storage_path(input_name, storage_dir)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
pub(crate) mod geoip;
|
||||
pub(crate) mod input_cache;
|
||||
pub(crate) mod playlist_watch;
|
||||
pub(crate) mod parser;
|
||||
pub(crate) mod processor;
|
||||
|
||||
|
||||
@@ -123,14 +123,12 @@ impl TVGuide {
|
||||
let src = "src".intern();
|
||||
for child in children {
|
||||
match child.name.as_ref() {
|
||||
EPG_TAG_DISPLAY_NAME => {
|
||||
if smart_match {
|
||||
EPG_TAG_DISPLAY_NAME if smart_match => {
|
||||
if let Some(name) = &child.value {
|
||||
tag.normalized_epg_ids
|
||||
.get_or_insert_with(Vec::new)
|
||||
.push(normalize_channel_name(name, &id_cache.smart_match_config).intern());
|
||||
}
|
||||
}
|
||||
}
|
||||
EPG_TAG_ICON => {
|
||||
if let Some(src) = child.get_attribute_value(&src) {
|
||||
|
||||
@@ -174,9 +174,7 @@ pub async fn parse_xtream(input: &ConfigInput,
|
||||
input.has_flag(ConfigInputFlags::XtreamLiveStreamWithoutExtension),
|
||||
);
|
||||
|
||||
// Re-implement the loop to add ordinal
|
||||
let mut ord_counter: u32 = 1;
|
||||
for stream in xtream_streams {
|
||||
for (ord_counter, stream) in (1_u32..).zip(xtream_streams) {
|
||||
let group = group_map.get_mut(&stream.get_category_id()).unwrap_or(&mut unknown_grp);
|
||||
let category_name = &group.category_name;
|
||||
let stream_url = create_xtream_url(xtream_cluster, url, username, password, &stream, live_stream_use_prefix, live_stream_without_extension);
|
||||
@@ -200,7 +198,6 @@ pub async fn parse_xtream(input: &ConfigInput,
|
||||
},
|
||||
};
|
||||
item.header.source_ordinal = ord_counter;
|
||||
ord_counter += 1;
|
||||
group.add(item);
|
||||
}
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ pub async fn process_group_watch(app_config: &Arc<AppConfig>, client: &reqwest::
|
||||
|
||||
let watch_filename = format!("{}/{}.bin", utils::sanitize_filename(target_name), utils::sanitize_filename(&pl.title));
|
||||
let cfg = app_config.config.load();
|
||||
match utils::get_file_path(&cfg.working_dir, Some(std::path::PathBuf::from(&watch_filename))) {
|
||||
match utils::get_file_path(&cfg.storage_dir, Some(std::path::PathBuf::from(&watch_filename))) {
|
||||
Some(path) => {
|
||||
let save_path = path.as_path();
|
||||
let mut changed = false;
|
||||
|
||||
@@ -50,13 +50,15 @@ use std::{
|
||||
collections::{HashMap, HashSet},
|
||||
path::PathBuf,
|
||||
sync::{Arc, Weak},
|
||||
time::Instant,
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
use tokio::{
|
||||
sync::{Mutex, OwnedRwLockWriteGuard, RwLock},
|
||||
task::JoinSet,
|
||||
};
|
||||
|
||||
const PLAYLIST_UPDATE_MAX_DURATION_SECS: u64 = 3600;
|
||||
|
||||
fn is_valid(pli: &PlaylistItem, filter: &Filter, match_as_ascii: bool) -> bool {
|
||||
let provider = ValueProvider { pli, match_as_ascii };
|
||||
filter.filter(&provider)
|
||||
@@ -324,10 +326,11 @@ async fn playlist_download_from_input(
|
||||
input: &ConfigInput,
|
||||
) -> PlaylistDownloadResult {
|
||||
let config = &*app_config.config.load();
|
||||
let working_dir = &config.working_dir;
|
||||
let storage_dir = &config.storage_dir;
|
||||
let download_input_type = input.get_download_input_type();
|
||||
|
||||
// Check Status
|
||||
let storage_path = input_cache::resolve_input_storage_path(working_dir, &input.name).await;
|
||||
let storage_path = input_cache::resolve_input_storage_path(storage_dir, &input.name).await;
|
||||
let mut status = input_cache::load_input_status(&storage_path);
|
||||
let cache_duration = input.cache_duration_seconds;
|
||||
|
||||
@@ -336,7 +339,7 @@ async fn playlist_download_from_input(
|
||||
let _ = std::fs::create_dir_all(&storage_path);
|
||||
}
|
||||
|
||||
let (clusters_to_download, fully_cached) = match input.input_type {
|
||||
let (clusters_to_download, fully_cached) = match download_input_type {
|
||||
InputType::Xtream => {
|
||||
let mut to_download = vec![];
|
||||
for c in XTREAM_CLUSTER {
|
||||
@@ -364,7 +367,7 @@ async fn playlist_download_from_input(
|
||||
return PlaylistDownloadResult::new(vec![], vec![], true, false);
|
||||
}
|
||||
|
||||
let (playlist, errors, persisted) = match input.input_type {
|
||||
let (playlist, errors, persisted) = match download_input_type {
|
||||
InputType::M3u => {
|
||||
let (p, e) = m3u::download_m3u_playlist(app_config, client, config, input).await;
|
||||
(p, e, false)
|
||||
@@ -381,7 +384,7 @@ async fn playlist_download_from_input(
|
||||
|
||||
// Update Status
|
||||
if errors.is_empty() {
|
||||
if let InputType::Xtream = input.input_type {
|
||||
if let InputType::Xtream = download_input_type {
|
||||
if let Some(clusters) = clusters_to_download {
|
||||
for c in clusters {
|
||||
input_cache::update_cluster_status(&mut status, &c.to_string(), ClusterState::Ok);
|
||||
@@ -401,7 +404,7 @@ async fn playlist_download_from_input(
|
||||
// We could mark specific clusters as failed if we knew which one failed.
|
||||
// For simplicity, if error, we don't update the timestamp (so it stays expired/invalid).
|
||||
// Or we mark as Failed.
|
||||
if let InputType::Xtream = input.input_type {
|
||||
if let InputType::Xtream = download_input_type {
|
||||
if let Some(clusters) = clusters_to_download {
|
||||
for c in clusters {
|
||||
// Optimistic: Only mark failed if we are sure?
|
||||
@@ -437,6 +440,7 @@ async fn process_source(
|
||||
continue;
|
||||
};
|
||||
if is_input_enabled(input, &ctx.user_targets) {
|
||||
let effective_input_type = input.get_download_input_type();
|
||||
source_downloaded = true;
|
||||
log_memory_snapshot(format!("source[{source_idx}] input '{}' before_download", input.name).as_str());
|
||||
|
||||
@@ -459,7 +463,7 @@ async fn process_source(
|
||||
};
|
||||
log_memory_snapshot(format!("source[{source_idx}] input '{}' after_download", input.name).as_str());
|
||||
|
||||
let tvguide = if input.input_type == InputType::Library {
|
||||
let tvguide = if effective_input_type == InputType::Library {
|
||||
None
|
||||
} else {
|
||||
download_input_epg(ctx, input, &mut error_list).await
|
||||
@@ -483,7 +487,7 @@ async fn process_source(
|
||||
let elapsed = start_time.elapsed().as_secs();
|
||||
input_stats.insert(
|
||||
input_name.clone(),
|
||||
create_input_stat(group_count, channel_count, errors.len(), input.input_type, input_name, elapsed),
|
||||
create_input_stat(group_count, channel_count, errors.len(), effective_input_type, input_name, elapsed),
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -547,8 +551,8 @@ async fn download_input_epg(
|
||||
) -> Option<TVGuide> {
|
||||
// Download epg for input
|
||||
let (tvguide, mut tvguide_errors) = if error_list.is_empty() {
|
||||
let working_dir = &ctx.config.config.load().working_dir;
|
||||
epg::get_xmltv(ctx, input, None, working_dir).await
|
||||
let storage_dir = &ctx.config.config.load().storage_dir;
|
||||
epg::get_xmltv(ctx, input, None, storage_dir).await
|
||||
} else {
|
||||
(None, vec![])
|
||||
};
|
||||
@@ -1018,10 +1022,10 @@ async fn playlist_probe(ctx: &PlaylistProcessingContext, target: &ConfigTarget,
|
||||
}
|
||||
|
||||
let input_name = fpl.input.name.clone();
|
||||
let input_type = fpl.input.input_type;
|
||||
let xtream_probe_handled = input_type.is_xtream() && target.get_xtream_output().is_some();
|
||||
let effective_input_type = fpl.input.get_download_input_type();
|
||||
let xtream_probe_handled = effective_input_type.is_xtream() && target.get_xtream_output().is_some();
|
||||
let live_probe_settings = if probe_live_enabled {
|
||||
get_live_probe_interval_settings(target, input_type, Some(opts)).map(|(delay, interval_secs)| {
|
||||
get_live_probe_interval_settings(target, effective_input_type, Some(opts)).map(|(delay, interval_secs)| {
|
||||
let interval_signed = i64::try_from(interval_secs).unwrap_or(i64::MAX);
|
||||
let cutoff_ts = chrono::Utc::now().timestamp().saturating_sub(interval_signed);
|
||||
(delay, interval_secs, cutoff_ts)
|
||||
@@ -1050,6 +1054,16 @@ async fn playlist_probe(ctx: &PlaylistProcessingContext, target: &ConfigTarget,
|
||||
if needs_live_probe(&item, cutoff_ts) {
|
||||
if let Some(provider_id) = provider_id_from_item(&item) {
|
||||
if queued_live_keys.insert(provider_id.clone()) {
|
||||
if log_enabled!(Level::Debug) {
|
||||
let last_probed = match item.header.additional_properties.as_ref() {
|
||||
Some(StreamProperties::Live(props)) => props.last_probed_timestamp,
|
||||
_ => None,
|
||||
};
|
||||
debug!(
|
||||
"[Task] Creating ProbeLive task for input {}: id={}, last_probed_ts={:?}, cutoff_ts={}, interval={}s, title=\"{}\"",
|
||||
input_name, provider_id, last_probed, cutoff_ts, interval_secs, item.header.title
|
||||
);
|
||||
}
|
||||
let task = UpdateTask::ProbeLive {
|
||||
id: provider_id,
|
||||
reason: ResolveReasonSet::from_variants(&[ResolveReason::Probe]),
|
||||
@@ -1092,7 +1106,7 @@ async fn playlist_probe(ctx: &PlaylistProcessingContext, target: &ConfigTarget,
|
||||
}
|
||||
|
||||
// For M3U, ID is a provider id; for Library, ID is UUID.
|
||||
let unique_id = if input_type == InputType::Library {
|
||||
let unique_id = if effective_input_type == InputType::Library {
|
||||
item.header.uuid.to_valid_uuid()
|
||||
} else {
|
||||
item.header.id.to_string()
|
||||
@@ -1104,6 +1118,10 @@ async fn playlist_probe(ctx: &PlaylistProcessingContext, target: &ConfigTarget,
|
||||
continue;
|
||||
}
|
||||
|
||||
debug!(
|
||||
"[Task] Creating ProbeStream task for input {}: scope={}, unique_id={}, item_type={:?}, title=\"{}\"",
|
||||
input_name, probe_scope, unique_id, item.header.item_type, item.header.title
|
||||
);
|
||||
let task = UpdateTask::ProbeStream {
|
||||
probe_scope,
|
||||
unique_id,
|
||||
@@ -1204,7 +1222,7 @@ async fn process_watch(
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
#[allow(clippy::too_many_arguments, clippy::too_many_lines)]
|
||||
pub async fn exec_processing(
|
||||
client: &reqwest::Client,
|
||||
app_config: Arc<AppConfig>,
|
||||
@@ -1219,6 +1237,7 @@ pub async fn exec_processing(
|
||||
pre_processed_inputs: Option<HashSet<Arc<str>>>,
|
||||
acquired_permit: Option<crate::api::model::UpdateGuardPermit>,
|
||||
) {
|
||||
let max_update_duration = Duration::from_secs(PLAYLIST_UPDATE_MAX_DURATION_SECS);
|
||||
let playlist_guard = if let Some(permit) = acquired_permit {
|
||||
Some(permit)
|
||||
} else if let Some(guard) = &update_guard {
|
||||
@@ -1237,7 +1256,15 @@ pub async fn exec_processing(
|
||||
|
||||
if playlist_guard.is_some() {
|
||||
if let Some(state) = app_state.as_ref() {
|
||||
sync_panel_api_exp_dates(state).await;
|
||||
if tokio::time::timeout(max_update_duration, sync_panel_api_exp_dates(state)).await.is_err() {
|
||||
error!(
|
||||
"Playlist update bootstrap timed out after {PLAYLIST_UPDATE_MAX_DURATION_SECS} secs while holding playlist lock",
|
||||
);
|
||||
if let Some(events) = event_manager.as_deref() {
|
||||
events.send_event(EventMessage::PlaylistUpdate(shared::model::PlaylistUpdateState::Failure));
|
||||
}
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1266,13 +1293,29 @@ pub async fn exec_processing(
|
||||
};
|
||||
|
||||
let start_time = Instant::now();
|
||||
let process_result = std::panic::AssertUnwindSafe(process_sources(&ctx)).catch_unwind().await;
|
||||
let Ok((stats, errors)) = process_result else {
|
||||
error!("Playlist processing panicked");
|
||||
if let Some(events) = event_manager.as_deref() {
|
||||
events.send_event(EventMessage::PlaylistUpdate(shared::model::PlaylistUpdateState::Failure));
|
||||
let process_result = tokio::time::timeout(
|
||||
max_update_duration,
|
||||
std::panic::AssertUnwindSafe(process_sources(&ctx)).catch_unwind(),
|
||||
)
|
||||
.await;
|
||||
let (stats, errors) = match process_result {
|
||||
Ok(Ok((stats, errors))) => (stats, errors),
|
||||
Ok(Err(_)) => {
|
||||
error!("Playlist processing panicked");
|
||||
if let Some(events) = event_manager.as_deref() {
|
||||
events.send_event(EventMessage::PlaylistUpdate(shared::model::PlaylistUpdateState::Failure));
|
||||
}
|
||||
return;
|
||||
}
|
||||
Err(_) => {
|
||||
error!(
|
||||
"Playlist processing timed out after {PLAYLIST_UPDATE_MAX_DURATION_SECS} secs while holding playlist lock",
|
||||
);
|
||||
if let Some(events) = event_manager.as_deref() {
|
||||
events.send_event(EventMessage::PlaylistUpdate(shared::model::PlaylistUpdateState::Failure));
|
||||
}
|
||||
return;
|
||||
}
|
||||
return;
|
||||
};
|
||||
log_memory_snapshot("exec_processing after_process_sources");
|
||||
|
||||
|
||||
@@ -41,7 +41,7 @@ pub async fn update_generic_stream_metadata(
|
||||
active_provider: &Arc<ActiveProviderManager>,
|
||||
active_handle: Option<&crate::api::model::ProviderHandle>,
|
||||
) -> Result<GenericProbeOutcome, TuliproxError> {
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let storage_dir = &app_config.config.load().storage_dir;
|
||||
|
||||
// Check if probing is enabled globally
|
||||
let ffprobe_enabled = app_config.is_ffprobe_enabled().await;
|
||||
@@ -50,7 +50,7 @@ pub async fn update_generic_stream_metadata(
|
||||
}
|
||||
|
||||
// Determine storage file path based on input type
|
||||
let storage_path = get_input_storage_path(&input.name, working_dir).await
|
||||
let storage_path = get_input_storage_path(&input.name, storage_dir).await
|
||||
.map_err(|e| shared::error::info_err!("Storage path error: {e}"))?;
|
||||
|
||||
let (db_path, storage_kind) = match input.input_type {
|
||||
|
||||
@@ -21,8 +21,8 @@ pub async fn update_live_stream_metadata(
|
||||
_active_handle: Option<&ProviderHandle>,
|
||||
_active_provider: &Arc<ActiveProviderManager>,
|
||||
) -> Result<Option<LiveStreamProperties>, TuliproxError> {
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let storage_path = get_input_storage_path(&input.name, working_dir).await
|
||||
let storage_dir = &app_config.config.load().storage_dir;
|
||||
let storage_path = get_input_storage_path(&input.name, storage_dir).await
|
||||
.map_err(|e| shared::error::info_err!("Storage path error: {e}"))?;
|
||||
|
||||
// Try to load existing info first to preserve data
|
||||
|
||||
@@ -207,6 +207,19 @@ fn queue_background_series_info(
|
||||
let reasons = check_resolve_reasons(resolve_options, resolve_tmdb_enabled, pli);
|
||||
|
||||
if !reasons.is_empty() {
|
||||
if log_enabled!(Level::Debug) {
|
||||
let has_details = pli.has_details();
|
||||
let (has_tmdb, has_date) = match pli.header.additional_properties.as_ref() {
|
||||
Some(StreamProperties::Series(props)) => {
|
||||
(props.tmdb.is_some(), props.release_date.is_some())
|
||||
}
|
||||
_ => (false, false),
|
||||
};
|
||||
debug!(
|
||||
"[Task] Creating ResolveSeries task for input {}: id={}, reasons={}, has_details={}, has_tmdb={}, has_date={}, title=\"{}\"",
|
||||
input_name_arc, provider_id, reasons, has_details, has_tmdb, has_date, pli.header.title
|
||||
);
|
||||
}
|
||||
let task = UpdateTask::ResolveSeries {
|
||||
id: provider_id,
|
||||
reason: reasons,
|
||||
@@ -233,8 +246,8 @@ async fn process_immediate_series_info(
|
||||
resolve_tmdb_enabled: bool,
|
||||
) -> Vec<PlaylistGroup> {
|
||||
let input = fpl.input;
|
||||
let working_dir = &ctx.config.config.load().working_dir;
|
||||
let storage_path = match get_input_storage_path(&input.name, working_dir).await {
|
||||
let storage_dir = &ctx.config.config.load().storage_dir;
|
||||
let storage_path = match get_input_storage_path(&input.name, storage_dir).await {
|
||||
Ok(path) => path,
|
||||
Err(err) => {
|
||||
error!("Can't resolve series, input storage directory for input '{}' failed: {err}", input.name);
|
||||
@@ -624,8 +637,8 @@ pub async fn update_series_metadata(
|
||||
db_query: Option<Arc<Mutex<BPlusTreeQuery<u32, XtreamPlaylistItem>>>>,
|
||||
tmdb_and_date_present_out: Option<&AtomicBool>,
|
||||
) -> Result<Option<SeriesStreamProperties>, TuliproxError> {
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let storage_path = get_input_storage_path(&input.name, working_dir)
|
||||
let storage_dir = &app_config.config.load().storage_dir;
|
||||
let storage_path = get_input_storage_path(&input.name, storage_dir)
|
||||
.await
|
||||
.map_err(|e| shared::error::info_err!("Storage path error: {e}"))?;
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ use crate::model::FetchedPlaylist;
|
||||
use crate::model::InputSource;
|
||||
use crate::model::{AppConfig, ConfigTarget};
|
||||
use crate::model::{ConfigInput, ConfigInputFlags};
|
||||
use crate::processing::input_cache::resolve_input_storage_path;
|
||||
use crate::processing::processor::playlist::PlaylistProcessingContext;
|
||||
use crate::processing::processor::{
|
||||
create_resolve_options_function_for_xtream_target, process_foreground_retry_once, select_cancel_token,
|
||||
@@ -12,7 +13,6 @@ use crate::processing::processor::{
|
||||
FOREGROUND_RETRY_BATCH_MAX_SIZE as RETRY_BATCH_MAX_SIZE,
|
||||
};
|
||||
use crate::ptt::ptt_parse_title;
|
||||
use crate::repository::get_input_storage_path;
|
||||
use crate::repository::persist_input_vod_info;
|
||||
use crate::repository::persist_input_vod_info_batch;
|
||||
use crate::repository::{xtream_get_file_path, BPlusTreeQuery};
|
||||
@@ -148,15 +148,9 @@ async fn process_immediate_vod_info(
|
||||
resolve_tmdb_enabled: bool,
|
||||
) {
|
||||
let input = fpl.input;
|
||||
let working_dir = &ctx.config.config.load().working_dir;
|
||||
let storage_dir = &ctx.config.config.load().storage_dir;
|
||||
|
||||
let storage_path = match get_input_storage_path(&input.name, working_dir).await {
|
||||
Ok(path) => path,
|
||||
Err(err) => {
|
||||
error!("Can't resolve vod, input storage directory for input '{}' failed: {err}", input.name);
|
||||
return;
|
||||
}
|
||||
};
|
||||
let storage_path = resolve_input_storage_path(storage_dir, &input.name).await;
|
||||
|
||||
// Keep an optional read query open and reopen lazily only when needed.
|
||||
let xtream_path = xtream_get_file_path(&storage_path, XtreamCluster::Video);
|
||||
@@ -447,6 +441,26 @@ fn queue_background_vod_info(
|
||||
let reasons = check_resolve_reasons(resolve_options, do_probe, resolve_tmdb_enabled, pli);
|
||||
|
||||
if !reasons.is_empty() {
|
||||
if log_enabled!(Level::Debug) {
|
||||
let has_details = pli.has_details();
|
||||
let (has_tmdb, has_date, has_video, has_audio) =
|
||||
match pli.header.additional_properties.as_ref() {
|
||||
Some(StreamProperties::Video(props)) => {
|
||||
let details = props.details.as_ref();
|
||||
(
|
||||
props.tmdb.is_some(),
|
||||
details.and_then(|d| d.release_date.as_ref()).is_some(),
|
||||
MediaQuality::is_valid_media_info(details.and_then(|d| d.video.as_deref())),
|
||||
MediaQuality::is_valid_media_info(details.and_then(|d| d.audio.as_deref())),
|
||||
)
|
||||
}
|
||||
_ => (false, false, false, false),
|
||||
};
|
||||
debug!(
|
||||
"[Task] Creating ResolveVod task for input {}: id={}, reasons={}, has_details={}, has_tmdb={}, has_date={}, has_video_info={}, has_audio_info={}, title=\"{}\"",
|
||||
input.name, provider_id, reasons, has_details, has_tmdb, has_date, has_video, has_audio, pli.header.title
|
||||
);
|
||||
}
|
||||
let task =
|
||||
UpdateTask::ResolveVod { id: provider_id, reason: reasons, delay: resolve_options.resolve_delay };
|
||||
mgr.queue_task_background(input.name.clone(), task);
|
||||
@@ -509,10 +523,8 @@ pub async fn update_vod_metadata(
|
||||
db_query: Option<Arc<Mutex<BPlusTreeQuery<u32, XtreamPlaylistItem>>>>,
|
||||
tmdb_resolved_out: Option<&AtomicBool>,
|
||||
) -> Result<Option<VideoStreamProperties>, TuliproxError> {
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let storage_path = get_input_storage_path(&input.name, working_dir)
|
||||
.await
|
||||
.map_err(|e| shared::error::info_err!("Storage path error: {e}"))?;
|
||||
let storage_dir = &app_config.config.load().storage_dir;
|
||||
let storage_path = resolve_input_storage_path(storage_dir, &input.name).await;
|
||||
|
||||
// Check if we should skip based on input options
|
||||
if input.has_flag(ConfigInputFlags::XtreamSkipVod) {
|
||||
|
||||
@@ -7,7 +7,7 @@ use log::{error, warn};
|
||||
use shared::error::{string_to_io_error, to_io_error, TuliproxError};
|
||||
use shared::info_err;
|
||||
use shared::model::{ConfigInputAliasDto, InputType};
|
||||
use shared::utils::{get_credentials_from_url, get_credentials_from_url_str, parse_timestamp, sanitize_sensitive_info, Internable, PROVIDER_SCHEME_PREFIX};
|
||||
use shared::utils::{get_credentials_from_url, get_credentials_from_url_str, parse_timestamp, sanitize_sensitive_info, Internable, BATCH_SCHEME_PREFIX, PROVIDER_SCHEME_PREFIX};
|
||||
use std::io;
|
||||
use std::io::{BufRead, Cursor, Error};
|
||||
use std::path::{Path, PathBuf};
|
||||
@@ -267,21 +267,23 @@ pub async fn csv_read_inputs(
|
||||
}
|
||||
|
||||
pub fn get_csv_file_path(file_uri: &str) -> Result<PathBuf, Error> {
|
||||
// Handle batch:// scheme: strip prefix and treat remainder as file path.
|
||||
if let Some(path_str) = file_uri.strip_prefix(BATCH_SCHEME_PREFIX) {
|
||||
let path = Path::new(path_str);
|
||||
return if path.is_absolute() {
|
||||
Ok(path.to_path_buf())
|
||||
} else {
|
||||
resolve_relative_path(path_str)
|
||||
};
|
||||
}
|
||||
let raw_path = Path::new(file_uri);
|
||||
if raw_path.is_absolute() {
|
||||
return Ok(raw_path.to_path_buf());
|
||||
}
|
||||
if let Ok(url) = file_uri.parse::<Url>() {
|
||||
if url.scheme() == "file" {
|
||||
match url.to_file_path() {
|
||||
Ok(path) => Ok(path),
|
||||
Err(()) => Err(string_to_io_error(format!("Could not open {file_uri}"))),
|
||||
}
|
||||
} else {
|
||||
Err(string_to_io_error(format!(
|
||||
"Only file:// is supported {file_uri}"
|
||||
)))
|
||||
}
|
||||
if let Ok(_url) = file_uri.parse::<Url>() {
|
||||
Err(string_to_io_error(format!(
|
||||
"Unsupported URL scheme for batch CSV, use batch:// instead: {file_uri}"
|
||||
)))
|
||||
} else {
|
||||
resolve_relative_path(file_uri)
|
||||
}
|
||||
|
||||
@@ -1747,7 +1747,7 @@ where
|
||||
let mut v = Vec::with_capacity(info.len());
|
||||
|
||||
let original_keys = std::mem::take(&mut keys);
|
||||
for (entry_key, entry_info) in original_keys.into_iter().zip(info.into_iter()) {
|
||||
for (entry_key, entry_info) in original_keys.into_iter().zip(info) {
|
||||
if entry_info.is_tombstone() {
|
||||
continue;
|
||||
}
|
||||
@@ -1854,7 +1854,7 @@ where
|
||||
let mut filtered_info: Vec<ValueInfo> = Vec::with_capacity(info.len());
|
||||
let mut last_packed_block: Option<(u64, Vec<u8>)> = None;
|
||||
let original_keys = std::mem::take(&mut keys);
|
||||
for (entry_key, entry_info) in original_keys.into_iter().zip(info.into_iter()) {
|
||||
for (entry_key, entry_info) in original_keys.into_iter().zip(info) {
|
||||
if entry_info.is_tombstone() {
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -181,8 +181,8 @@ async fn persist_m3u_playlist_as_text(
|
||||
let Some(filename) = target_output.filename.as_ref() else {
|
||||
return Ok(());
|
||||
};
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let Some(m3u_filename) = utils::get_file_path(working_dir, Some(PathBuf::from(filename))) else {
|
||||
let cfg = app_config.config.load();
|
||||
let Some(m3u_filename) = utils::get_file_path(&cfg.storage_dir, Some(PathBuf::from(filename))) else {
|
||||
return Ok(());
|
||||
};
|
||||
|
||||
@@ -375,8 +375,8 @@ pub async fn iter_raw_m3u_input_playlist(
|
||||
input: &ConfigInput,
|
||||
cluster: Option<XtreamCluster>,
|
||||
) -> Option<Box<dyn Stream<Item=Result<M3uPlaylistItem, TuliproxError>> + Send + Unpin>> {
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let storage_path = get_input_storage_path(&input.name, working_dir).await.ok()?;
|
||||
let cfg = app_config.config.load();
|
||||
let storage_path = get_input_storage_path(&input.name, &cfg.storage_dir).await.ok()?;
|
||||
let m3u_path = get_input_m3u_playlist_file_path(&storage_path, &input.name);
|
||||
|
||||
iter_raw_m3u_playlist::<u32, Arc<str>>(app_config, &m3u_path, cluster).await
|
||||
|
||||
@@ -17,6 +17,7 @@ mod library_repository;
|
||||
mod sorted_index;
|
||||
mod alias_repository;
|
||||
mod playlist_stream;
|
||||
mod provider_dns_repository;
|
||||
|
||||
pub use storage::*;
|
||||
pub use target_id_mapping::*;
|
||||
@@ -36,6 +37,7 @@ pub use alias_repository::*;
|
||||
pub use playlist_source::*;
|
||||
pub use library_repository::*;
|
||||
pub use playlist_stream::*;
|
||||
pub use provider_dns_repository::*;
|
||||
|
||||
#[cfg(test)]
|
||||
mod bplustree_stress;
|
||||
|
||||
@@ -371,28 +371,21 @@ pub async fn persist_input_playlist(app_config: &Arc<AppConfig>, input: &ConfigI
|
||||
return (playlist, None);
|
||||
}
|
||||
playlist.iter_mut().for_each(PlaylistGroup::on_load);
|
||||
let cfg = app_config.config.load();
|
||||
let storage_path = match get_input_storage_path(&input.name, &cfg.storage_dir).await {
|
||||
Ok(storage_path) => storage_path,
|
||||
Err(err) => {
|
||||
return (playlist, Some(info_err!("Error creating input storage directory for input '{}' failed: {err}", input.name)));
|
||||
}
|
||||
};
|
||||
|
||||
match input.input_type {
|
||||
match input.get_download_input_type() {
|
||||
InputType::Xtream | InputType::XtreamBatch => {
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let storage_path = match get_input_storage_path(&input.name, working_dir).await {
|
||||
Ok(storage_path) => storage_path,
|
||||
Err(err) => {
|
||||
return (playlist, Some(info_err!("Error creating input storage directory for input '{}' failed: {err}", input.name)));
|
||||
}
|
||||
};
|
||||
persist_input_xtream_playlist(app_config, &storage_path, playlist).await
|
||||
}
|
||||
|
||||
InputType::M3u | InputType::M3uBatch => {
|
||||
// Persist M3U
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let storage_path = match get_input_storage_path(&input.name, working_dir).await {
|
||||
Ok(storage_path) => storage_path,
|
||||
Err(err) => {
|
||||
return (playlist, Some(info_err!("Error creating input storage directory for input '{}' failed: {err}", input.name)));
|
||||
}
|
||||
};
|
||||
let file_path = get_input_m3u_playlist_file_path(&storage_path, &input.name);
|
||||
if let Err(err) = persist_input_m3u_playlist(app_config, &file_path, &playlist).await {
|
||||
return (playlist, Some(err));
|
||||
@@ -401,13 +394,6 @@ pub async fn persist_input_playlist(app_config: &Arc<AppConfig>, input: &ConfigI
|
||||
}
|
||||
InputType::Library => {
|
||||
// Persist local library playlist
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let storage_path = match get_input_storage_path(&input.name, working_dir).await {
|
||||
Ok(storage_path) => storage_path,
|
||||
Err(err) => {
|
||||
return (playlist, Some(info_err!("Error creating input storage directory for input '{}' failed: {err}", input.name)));
|
||||
}
|
||||
};
|
||||
let file_path = get_input_local_library_playlist_file_path(&storage_path, &input.name);
|
||||
let (playlist, result) = persist_input_library_playlist(app_config, &file_path, playlist).await;
|
||||
if let Err(err) = result {
|
||||
@@ -420,13 +406,12 @@ pub async fn persist_input_playlist(app_config: &Arc<AppConfig>, input: &ConfigI
|
||||
|
||||
pub async fn load_input_playlist(ctx: &PlaylistProcessingContext, input: &ConfigInput, clusters: Option<&[XtreamCluster]>) -> Result<Box<dyn PlaylistSource>, TuliproxError> {
|
||||
let app_config = &ctx.config;
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let storage_path = get_input_storage_path(&input.name, working_dir).await
|
||||
let cfg = app_config.config.load();
|
||||
let storage_path = get_input_storage_path(&input.name, &cfg.storage_dir).await
|
||||
.map_err(|e| info_err!("Error getting input path: {e}"))?;
|
||||
let disk_based_processing = cfg.disk_based_processing;
|
||||
|
||||
let disk_based_processing = app_config.config.load().disk_based_processing;
|
||||
|
||||
match input.input_type {
|
||||
match input.get_download_input_type() {
|
||||
InputType::Xtream | InputType::XtreamBatch => {
|
||||
if disk_based_processing {
|
||||
Ok(Box::new(XtreamDiskPlaylistSource::new(app_config, &storage_path).await))
|
||||
|
||||
@@ -0,0 +1,434 @@
|
||||
use indexmap::IndexMap;
|
||||
use log::{debug, info, warn};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use shared::utils::DNS_RESOLVED_FILE;
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::io::ErrorKind;
|
||||
use std::net::IpAddr;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
use tokio::fs;
|
||||
use tokio::sync::mpsc;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
use crate::api::model::AppState;
|
||||
use crate::model::ConfigProvider;
|
||||
|
||||
/// All providers' resolved DNS data, keyed by provider name → hostname → IPs.
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
pub struct DnsResolvedStore {
|
||||
#[serde(flatten)]
|
||||
pub providers: HashMap<String, IndexMap<String, Vec<IpAddr>>>,
|
||||
}
|
||||
|
||||
static DNS_WRITER_GENERATION: AtomicU64 = AtomicU64::new(0);
|
||||
const DNS_WRITER_FLUSH_INTERVAL_SECS: u64 = 2;
|
||||
const DNS_WRITER_FLUSH_BATCH_THRESHOLD: usize = 32;
|
||||
|
||||
pub fn next_dns_writer_generation() -> u64 { DNS_WRITER_GENERATION.fetch_add(1, Ordering::SeqCst) + 1 }
|
||||
|
||||
fn is_dns_writer_generation_current(generation: u64) -> bool {
|
||||
DNS_WRITER_GENERATION.load(Ordering::SeqCst) == generation
|
||||
}
|
||||
|
||||
pub fn dns_resolved_file_path(storage_dir: &str) -> PathBuf {
|
||||
PathBuf::from(storage_dir).join(DNS_RESOLVED_FILE)
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum DnsResolvedStoreLoadError {
|
||||
Read(std::io::Error),
|
||||
Parse(serde_json::Error),
|
||||
}
|
||||
|
||||
impl std::fmt::Display for DnsResolvedStoreLoadError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Self::Read(err) => write!(f, "read failed: {err}"),
|
||||
Self::Parse(err) => write!(f, "parse failed: {err}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn load_dns_resolved_store_from_path(path: &Path) -> Result<Option<DnsResolvedStore>, DnsResolvedStoreLoadError> {
|
||||
match fs::read_to_string(path).await {
|
||||
Ok(data) => serde_json::from_str(&data).map(Some).map_err(DnsResolvedStoreLoadError::Parse),
|
||||
Err(err) if err.kind() == ErrorKind::NotFound => Ok(None),
|
||||
Err(err) => Err(DnsResolvedStoreLoadError::Read(err)),
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn persist_dns_resolved_store(path: &Path, store: &DnsResolvedStore) -> Result<(), String> {
|
||||
let json = match serde_json::to_string_pretty(store) {
|
||||
Ok(value) => value,
|
||||
Err(err) => return Err(format!("serialize failed: {err}")),
|
||||
};
|
||||
|
||||
if let Some(parent) = path.parent() {
|
||||
fs::create_dir_all(parent)
|
||||
.await
|
||||
.map_err(|err| format!("create parent dir '{}' failed: {err}", parent.display()))?;
|
||||
}
|
||||
|
||||
let tmp_path = path.with_extension("json.tmp");
|
||||
fs::write(&tmp_path, json)
|
||||
.await
|
||||
.map_err(|err| format!("write temp file '{}' failed: {err}", tmp_path.display()))?;
|
||||
|
||||
if let Err(err) = fs::rename(&tmp_path, path).await {
|
||||
#[cfg(windows)]
|
||||
{
|
||||
if fs::remove_file(path).await.is_ok() && fs::rename(&tmp_path, path).await.is_ok() {
|
||||
debug!(
|
||||
"Persisted DNS resolved store to '{}' (providers={})",
|
||||
path.display(),
|
||||
store.providers.len()
|
||||
);
|
||||
return Ok(());
|
||||
}
|
||||
}
|
||||
let _ = fs::remove_file(&tmp_path).await;
|
||||
return Err(format!(
|
||||
"rename temp file '{}' → '{}' failed: {err}",
|
||||
tmp_path.display(),
|
||||
path.display()
|
||||
));
|
||||
}
|
||||
|
||||
debug!(
|
||||
"Persisted DNS resolved store to '{}' (providers={})",
|
||||
path.display(),
|
||||
store.providers.len()
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Load persisted DNS resolved data and seed the in-memory caches for all providers.
|
||||
/// Called at startup and after config reloads, before DNS background tasks run.
|
||||
pub async fn load_persisted_dns_resolved(app_state: &std::sync::Arc<crate::api::model::AppState>) {
|
||||
let storage_dir = app_state.app_config.config.load().storage_dir.clone();
|
||||
let path = dns_resolved_file_path(&storage_dir);
|
||||
|
||||
let store = match load_dns_resolved_store_from_path(&path).await {
|
||||
Ok(Some(store)) => store,
|
||||
Ok(None) => {
|
||||
debug!("No persisted DNS resolved file found at '{}'", path.display());
|
||||
return;
|
||||
}
|
||||
Err(err) => {
|
||||
warn!("Failed to load persisted DNS resolved file '{}': {err}", path.display());
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let sources = app_state.app_config.sources.load();
|
||||
let mut seeded_count = 0usize;
|
||||
for (provider_name, hosts) in &store.providers {
|
||||
let Some(provider) = sources.get_provider_by_name(provider_name) else {
|
||||
debug!("Persisted DNS resolved data for unknown provider '{provider_name}', skipping");
|
||||
continue;
|
||||
};
|
||||
if !provider.get_dns_config().is_some_and(|cfg| cfg.enabled) {
|
||||
debug!("Persisted DNS resolved data for disabled-dns provider '{provider_name}', skipping");
|
||||
continue;
|
||||
}
|
||||
for (host, ips) in hosts {
|
||||
if !ips.is_empty() {
|
||||
// Only seed if the cache is currently empty for this host (don't overwrite runtime data).
|
||||
if provider.dns_cache.ip_count(host) == 0 {
|
||||
provider.dns_cache.store_resolved(host, ips.clone());
|
||||
seeded_count += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if seeded_count > 0 {
|
||||
info!("Seeded {seeded_count} host(s) from persisted DNS resolved file '{}'", path.display());
|
||||
}
|
||||
}
|
||||
|
||||
fn prune_store_to_runtime_enabled_providers(app_state: &Arc<AppState>, store: &mut DnsResolvedStore) -> usize {
|
||||
let sources = app_state.app_config.sources.load();
|
||||
let enabled_provider_names = sources
|
||||
.provider
|
||||
.iter()
|
||||
.filter(|provider| provider.get_dns_config().is_some_and(|cfg| cfg.enabled))
|
||||
.map(|provider| provider.name.to_string())
|
||||
.collect::<HashSet<String>>();
|
||||
prune_store_to_enabled_provider_names(store, &enabled_provider_names)
|
||||
}
|
||||
|
||||
fn prune_store_to_enabled_provider_names(
|
||||
store: &mut DnsResolvedStore,
|
||||
enabled_provider_names: &HashSet<String>,
|
||||
) -> usize {
|
||||
let before = store.providers.len();
|
||||
store
|
||||
.providers
|
||||
.retain(|provider_name, _| enabled_provider_names.contains(provider_name));
|
||||
before.saturating_sub(store.providers.len())
|
||||
}
|
||||
|
||||
fn is_provider_dns_enabled_in_runtime(app_state: &Arc<AppState>, provider_name: &str) -> bool {
|
||||
let sources = app_state.app_config.sources.load();
|
||||
sources
|
||||
.get_provider_by_name(provider_name)
|
||||
.is_some_and(|provider| provider.get_dns_config().is_some_and(|cfg| cfg.enabled))
|
||||
}
|
||||
|
||||
pub async fn prune_persisted_dns_resolved_to_runtime(app_state: &Arc<AppState>) {
|
||||
let storage_dir = app_state.app_config.config.load().storage_dir.clone();
|
||||
let path = dns_resolved_file_path(&storage_dir);
|
||||
let mut store = match load_dns_resolved_store_from_path(&path).await {
|
||||
Ok(Some(existing)) => existing,
|
||||
Ok(None) => return,
|
||||
Err(err) => {
|
||||
warn!(
|
||||
"Failed to load DNS resolved store for runtime-prune '{}': {err}",
|
||||
path.display()
|
||||
);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let removed = prune_store_to_runtime_enabled_providers(app_state, &mut store);
|
||||
if removed == 0 {
|
||||
return;
|
||||
}
|
||||
|
||||
if let Err(err) = persist_dns_resolved_store(&path, &store).await {
|
||||
warn!(
|
||||
"Failed to persist DNS resolved store after runtime-prune '{}': {err}",
|
||||
path.display()
|
||||
);
|
||||
} else {
|
||||
info!(
|
||||
"Pruned {removed} stale provider(s) from DNS resolved store '{}'",
|
||||
path.display()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct DnsResolvedWriteUpdate {
|
||||
pub provider_name: String,
|
||||
pub resolved: IndexMap<String, Vec<IpAddr>>,
|
||||
}
|
||||
|
||||
pub type DnsResolvedWriteTx = mpsc::Sender<DnsResolvedWriteUpdate>;
|
||||
|
||||
#[allow(clippy::too_many_lines)]
|
||||
pub fn spawn_dns_resolved_writer(
|
||||
app_state: Arc<AppState>,
|
||||
cancel: CancellationToken,
|
||||
generation: u64,
|
||||
) -> DnsResolvedWriteTx {
|
||||
let storage_dir = app_state.app_config.config.load().storage_dir.clone();
|
||||
let path = dns_resolved_file_path(&storage_dir);
|
||||
let (tx, mut rx) = mpsc::channel::<DnsResolvedWriteUpdate>(256);
|
||||
|
||||
tokio::spawn(async move {
|
||||
let mut store = match load_dns_resolved_store_from_path(&path).await {
|
||||
Ok(Some(existing)) => existing,
|
||||
Ok(None) => DnsResolvedStore::default(),
|
||||
Err(err) => {
|
||||
warn!("Failed to load existing DNS resolved store '{}': {err}", path.display());
|
||||
DnsResolvedStore::default()
|
||||
}
|
||||
};
|
||||
let removed_on_start = prune_store_to_runtime_enabled_providers(&app_state, &mut store);
|
||||
if removed_on_start > 0 {
|
||||
info!(
|
||||
"Pruned {removed_on_start} stale provider(s) from DNS resolved store on writer start '{}'",
|
||||
path.display()
|
||||
);
|
||||
if let Err(err) = persist_dns_resolved_store(&path, &store).await {
|
||||
warn!(
|
||||
"Failed to persist DNS resolved store after startup prune '{}': {err}",
|
||||
path.display()
|
||||
);
|
||||
}
|
||||
}
|
||||
let mut dirty = false;
|
||||
let mut dirty_updates = 0usize;
|
||||
let mut flush_before_exit = false;
|
||||
let mut flush_timer = std::pin::pin!(tokio::time::sleep(Duration::from_secs(
|
||||
DNS_WRITER_FLUSH_INTERVAL_SECS,
|
||||
)));
|
||||
let mut flush_timer_active = false;
|
||||
|
||||
loop {
|
||||
if !is_dns_writer_generation_current(generation) {
|
||||
debug!(
|
||||
"Stopping stale DNS resolved writer generation {} for '{}'",
|
||||
generation,
|
||||
path.display()
|
||||
);
|
||||
break;
|
||||
}
|
||||
tokio::select! {
|
||||
() = cancel.cancelled() => {
|
||||
debug!("Stopping DNS resolved writer task for '{}'", path.display());
|
||||
flush_before_exit = true;
|
||||
break;
|
||||
}
|
||||
maybe_update = rx.recv() => {
|
||||
let Some(update) = maybe_update else {
|
||||
debug!("DNS resolved writer channel closed for '{}'", path.display());
|
||||
flush_before_exit = true;
|
||||
break;
|
||||
};
|
||||
if !is_dns_writer_generation_current(generation) {
|
||||
debug!(
|
||||
"Dropping update for stale DNS resolved writer generation {} on '{}'",
|
||||
generation,
|
||||
path.display()
|
||||
);
|
||||
break;
|
||||
}
|
||||
|
||||
if update.resolved.is_empty() || !is_provider_dns_enabled_in_runtime(&app_state, update.provider_name.as_str()) {
|
||||
store.providers.remove(update.provider_name.as_str());
|
||||
} else {
|
||||
store.providers.insert(update.provider_name, update.resolved);
|
||||
}
|
||||
|
||||
dirty = true;
|
||||
dirty_updates = dirty_updates.saturating_add(1);
|
||||
|
||||
if dirty_updates >= DNS_WRITER_FLUSH_BATCH_THRESHOLD {
|
||||
let _ = prune_store_to_runtime_enabled_providers(&app_state, &mut store);
|
||||
if let Err(err) = persist_dns_resolved_store(&path, &store).await {
|
||||
warn!("Failed to persist DNS resolved store '{}': {err}", path.display());
|
||||
} else {
|
||||
dirty = false;
|
||||
dirty_updates = 0;
|
||||
flush_timer_active = false;
|
||||
}
|
||||
} else if !flush_timer_active {
|
||||
flush_timer.as_mut().reset(
|
||||
tokio::time::Instant::now() + Duration::from_secs(DNS_WRITER_FLUSH_INTERVAL_SECS),
|
||||
);
|
||||
flush_timer_active = true;
|
||||
}
|
||||
}
|
||||
() = &mut flush_timer, if flush_timer_active => {
|
||||
flush_timer_active = false;
|
||||
if dirty {
|
||||
let _ = prune_store_to_runtime_enabled_providers(&app_state, &mut store);
|
||||
if let Err(err) = persist_dns_resolved_store(&path, &store).await {
|
||||
warn!("Failed to persist DNS resolved store '{}': {err}", path.display());
|
||||
} else {
|
||||
dirty = false;
|
||||
dirty_updates = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if flush_before_exit && dirty {
|
||||
let _ = prune_store_to_runtime_enabled_providers(&app_state, &mut store);
|
||||
if let Err(err) = persist_dns_resolved_store(&path, &store).await {
|
||||
warn!(
|
||||
"Failed to persist DNS resolved store during writer shutdown '{}': {err}",
|
||||
path.display()
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
tx
|
||||
}
|
||||
|
||||
pub async fn queue_provider_resolved_snapshot(writer_tx: &DnsResolvedWriteTx, provider: &Arc<ConfigProvider>) {
|
||||
let update = DnsResolvedWriteUpdate {
|
||||
provider_name: provider.name.to_string(),
|
||||
resolved: provider.snapshot_resolved_ordered(),
|
||||
};
|
||||
if let Err(err) = writer_tx.send(update).await {
|
||||
warn!("Failed to queue DNS resolved update for provider '{}': {err}", provider.name);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{
|
||||
is_dns_writer_generation_current, load_dns_resolved_store_from_path, next_dns_writer_generation,
|
||||
persist_dns_resolved_store, prune_store_to_enabled_provider_names, DnsResolvedStore, DnsResolvedStoreLoadError,
|
||||
};
|
||||
use indexmap::IndexMap;
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::net::IpAddr;
|
||||
|
||||
#[tokio::test]
|
||||
async fn load_dns_resolved_store_reports_parse_error() {
|
||||
let dir = tempfile::tempdir().expect("temp dir should be created");
|
||||
let path = dir.path().join("provider_dns_resolved.json");
|
||||
tokio::fs::write(&path, "{ not-json").await.expect("test file should be written");
|
||||
|
||||
let result = load_dns_resolved_store_from_path(&path).await;
|
||||
assert!(matches!(result, Err(DnsResolvedStoreLoadError::Parse(_))));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn persist_and_load_dns_resolved_store_roundtrip() {
|
||||
let dir = tempfile::tempdir().expect("temp dir should be created");
|
||||
let path = dir.path().join("provider_dns_resolved.json");
|
||||
let mut providers = HashMap::new();
|
||||
providers.insert(
|
||||
"provider-a".to_string(),
|
||||
IndexMap::from([(
|
||||
"example.com".to_string(),
|
||||
vec!["203.0.113.10".parse::<IpAddr>().expect("valid ip")],
|
||||
)]),
|
||||
);
|
||||
let store = DnsResolvedStore { providers };
|
||||
|
||||
persist_dns_resolved_store(&path, &store).await.expect("persist should succeed");
|
||||
let loaded = load_dns_resolved_store_from_path(&path)
|
||||
.await
|
||||
.expect("load should succeed")
|
||||
.expect("store should exist");
|
||||
|
||||
assert_eq!(loaded.providers.len(), 1);
|
||||
assert_eq!(
|
||||
loaded
|
||||
.providers
|
||||
.get("provider-a")
|
||||
.and_then(|hosts| hosts.get("example.com")),
|
||||
Some(&vec!["203.0.113.10".parse::<IpAddr>().expect("valid ip")])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generation_switch_marks_previous_writer_as_stale() {
|
||||
let old_generation = next_dns_writer_generation();
|
||||
let new_generation = next_dns_writer_generation();
|
||||
assert_ne!(old_generation, new_generation);
|
||||
assert!(!is_dns_writer_generation_current(old_generation));
|
||||
assert!(is_dns_writer_generation_current(new_generation));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prune_store_to_enabled_provider_names_removes_stale_entries() {
|
||||
let mut providers = HashMap::new();
|
||||
providers.insert(
|
||||
"provider-a".to_string(),
|
||||
IndexMap::from([("example.com".to_string(), vec!["203.0.113.10".parse::<IpAddr>().expect("valid ip")])]),
|
||||
);
|
||||
providers.insert(
|
||||
"provider-b".to_string(),
|
||||
IndexMap::from([("example.org".to_string(), vec!["203.0.113.20".parse::<IpAddr>().expect("valid ip")])]),
|
||||
);
|
||||
let mut store = DnsResolvedStore { providers };
|
||||
let enabled = HashSet::from(["provider-a".to_string()]);
|
||||
|
||||
let removed = prune_store_to_enabled_provider_names(&mut store, &enabled);
|
||||
assert_eq!(removed, 1);
|
||||
assert!(store.providers.contains_key("provider-a"));
|
||||
assert!(!store.providers.contains_key("provider-b"));
|
||||
}
|
||||
}
|
||||
@@ -25,7 +25,7 @@ pub async fn ensure_target_storage_path(cfg: &Config, target_name: &str) -> Resu
|
||||
}
|
||||
|
||||
pub fn get_target_storage_path(cfg: &Config, target_name: &str) -> Option<PathBuf> {
|
||||
utils::get_file_path(&cfg.working_dir, Some(std::path::PathBuf::from(target_name.replace(' ', "_"))))
|
||||
utils::get_file_path(&cfg.storage_dir, Some(std::path::PathBuf::from(target_name.replace(' ', "_"))))
|
||||
}
|
||||
|
||||
pub fn sanitize_name(name: &str) -> String {
|
||||
@@ -34,27 +34,27 @@ pub fn sanitize_name(name: &str) -> String {
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub fn build_input_storage_path(input_name: &str, working_dir: &str) -> PathBuf {
|
||||
pub fn build_input_storage_path(input_name: &str, storage_dir: &str) -> PathBuf {
|
||||
let sanitized_name: String = sanitize_name(input_name);
|
||||
let name = concat_string!(cap = 6 + sanitized_name.len(); "input_", &sanitized_name);
|
||||
Path::new(working_dir).join(name)
|
||||
Path::new(storage_dir).join(name)
|
||||
}
|
||||
|
||||
pub async fn get_input_storage_path(input_name: &str, working_dir: &str) -> std::io::Result<PathBuf> {
|
||||
let path = build_input_storage_path(input_name, working_dir);
|
||||
pub async fn get_input_storage_path(input_name: &str, storage_dir: &str) -> std::io::Result<PathBuf> {
|
||||
let path = build_input_storage_path(input_name, storage_dir);
|
||||
// Create the directory and return the path or propagate the error
|
||||
tokio::fs::create_dir_all(&path).await.map(|()| path)
|
||||
}
|
||||
|
||||
pub async fn ensure_input_storage_path(cfg: &Config, input_name: &str) -> Result<PathBuf, TuliproxError> {
|
||||
get_input_storage_path(input_name, &cfg.working_dir).await
|
||||
get_input_storage_path(input_name, &cfg.storage_dir).await
|
||||
.map_err(|err| {
|
||||
notify_err!("Failed to save input data, can't create directory for input {input_name}: {err}")
|
||||
})
|
||||
}
|
||||
|
||||
pub fn get_geoip_path(working_dir: &str) -> PathBuf {
|
||||
Path::new(working_dir).join("geoip.db")
|
||||
pub fn get_geoip_path(storage_dir: &str) -> PathBuf {
|
||||
Path::new(storage_dir).join("geoip.db")
|
||||
}
|
||||
|
||||
pub fn get_file_path_for_db_index(db_path: &Path) -> PathBuf {
|
||||
|
||||
@@ -850,7 +850,7 @@ pub async fn write_strm_playlist(
|
||||
|
||||
let config = app_config.config.load();
|
||||
let Some(root_path) = crate::utils::get_file_path(
|
||||
&config.working_dir,
|
||||
&config.storage_dir,
|
||||
Some(std::path::PathBuf::from(&target_output.directory)),
|
||||
) else {
|
||||
return info_err_res!("Failed to get file path for {}",target_output.directory);
|
||||
|
||||
@@ -517,7 +517,9 @@ mod tests {
|
||||
hdhomerun: Arc::new(ArcSwapAny::default()),
|
||||
api_proxy: Arc::new(ArcSwapAny::default()),
|
||||
paths: Arc::new(ArcSwap::from(Arc::new(ConfigPaths {
|
||||
home_path: String::new(),
|
||||
config_path: temp_dir().to_string_lossy().to_string(),
|
||||
storage_path: temp_dir().to_string_lossy().to_string(),
|
||||
config_file_path: String::new(),
|
||||
sources_file_path: String::new(),
|
||||
mapping_file_path: None,
|
||||
|
||||
@@ -654,8 +654,8 @@ pub async fn iter_raw_xtream_target_playlist(app_config: &AppConfig, target: &Co
|
||||
|
||||
pub async fn iter_raw_xtream_input_playlist(app_config: &AppConfig, input: &ConfigInput, cluster: XtreamCluster) -> Option<Box<dyn Stream<Item = XtreamPlaylistItem> + Send + Unpin>> {
|
||||
let config = app_config.config.load();
|
||||
let working_dir = &config.working_dir;
|
||||
let storage_path = get_input_storage_path(&input.name, working_dir).await.ok()?;
|
||||
let storage_dir = &config.storage_dir;
|
||||
let storage_path = get_input_storage_path(&input.name, storage_dir).await.ok()?;
|
||||
let xtream_path = xtream_get_file_path(&storage_path, cluster);
|
||||
|
||||
iter_raw_xtream_playlist(app_config, &xtream_path).await
|
||||
@@ -1211,15 +1211,14 @@ fn needs_preserved_stream_property_merge(
|
||||
let preserve_info_details = old_stream_props.has_details() && !needs_update_info_details(new_stream_props, old_stream_props);
|
||||
|
||||
match (new_stream_props, old_stream_props) {
|
||||
(StreamProperties::Video(v_new), StreamProperties::Video(v_old))
|
||||
if preserve_info_details && v_old.details.is_some() =>
|
||||
{
|
||||
v_new.details != v_old.details
|
||||
(StreamProperties::Video(v_new), StreamProperties::Video(v_old)) => {
|
||||
(preserve_info_details && v_old.details.is_some() && v_new.details != v_old.details)
|
||||
|| (v_new.tmdb.is_none() && v_old.tmdb.is_some())
|
||||
}
|
||||
(StreamProperties::Series(s_new), StreamProperties::Series(s_old))
|
||||
if preserve_info_details && s_old.details.is_some() =>
|
||||
{
|
||||
s_new.details != s_old.details
|
||||
(StreamProperties::Series(s_new), StreamProperties::Series(s_old)) => {
|
||||
(preserve_info_details && s_old.details.is_some() && s_new.details != s_old.details)
|
||||
|| (s_new.tmdb.is_none() && s_old.tmdb.is_some())
|
||||
|| (s_new.release_date.is_none() && s_old.release_date.is_some())
|
||||
}
|
||||
(StreamProperties::Live(l_new), StreamProperties::Live(l_old)) => {
|
||||
(l_new.video.is_none() && l_old.video.is_some())
|
||||
@@ -1245,26 +1244,41 @@ pub(crate) fn merge_preserved_stream_properties(
|
||||
old_stream_props.has_details() && !needs_update_info_details(new_stream_props, old_stream_props);
|
||||
|
||||
match (new_stream_props, old_stream_props) {
|
||||
(StreamProperties::Video(v_new), StreamProperties::Video(v_old))
|
||||
if preserve_info_details && v_old.details.is_some() =>
|
||||
{
|
||||
if v_new.details == v_old.details {
|
||||
false
|
||||
} else {
|
||||
v_new.details.clone_from(&v_old.details);
|
||||
true
|
||||
}
|
||||
(StreamProperties::Video(v_new), StreamProperties::Video(v_old)) => {
|
||||
let mut changed = false;
|
||||
|
||||
if preserve_info_details && v_old.details.is_some() && v_new.details != v_old.details {
|
||||
v_new.details.clone_from(&v_old.details);
|
||||
changed = true;
|
||||
}
|
||||
(StreamProperties::Series(s_new), StreamProperties::Series(s_old))
|
||||
if preserve_info_details && s_old.details.is_some() =>
|
||||
{
|
||||
if s_new.details == s_old.details {
|
||||
false
|
||||
} else {
|
||||
s_new.details.clone_from(&s_old.details);
|
||||
true
|
||||
}
|
||||
|
||||
if v_new.tmdb.is_none() && v_old.tmdb.is_some() {
|
||||
v_new.tmdb = v_old.tmdb;
|
||||
changed = true;
|
||||
}
|
||||
|
||||
changed
|
||||
}
|
||||
(StreamProperties::Series(s_new), StreamProperties::Series(s_old)) => {
|
||||
let mut changed = false;
|
||||
|
||||
if preserve_info_details && s_old.details.is_some() && s_new.details != s_old.details {
|
||||
s_new.details.clone_from(&s_old.details);
|
||||
changed = true;
|
||||
}
|
||||
|
||||
if s_new.tmdb.is_none() && s_old.tmdb.is_some() {
|
||||
s_new.tmdb = s_old.tmdb;
|
||||
changed = true;
|
||||
}
|
||||
|
||||
if s_new.release_date.is_none() && s_old.release_date.is_some() {
|
||||
s_new.release_date.clone_from(&s_old.release_date);
|
||||
changed = true;
|
||||
}
|
||||
|
||||
changed
|
||||
}
|
||||
(StreamProperties::Live(l_new), StreamProperties::Live(l_old)) => {
|
||||
let mut changed = false;
|
||||
|
||||
@@ -1586,6 +1600,49 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn merge_preserves_missing_video_tmdb() {
|
||||
let mut new_props = StreamProperties::Video(Box::new(VideoStreamProperties {
|
||||
tmdb: None,
|
||||
..VideoStreamProperties::default()
|
||||
}));
|
||||
let old_props = StreamProperties::Video(Box::new(VideoStreamProperties {
|
||||
tmdb: Some(317981),
|
||||
..VideoStreamProperties::default()
|
||||
}));
|
||||
|
||||
let changed = merge_preserved_stream_properties(&mut new_props, &old_props);
|
||||
assert!(changed);
|
||||
match new_props {
|
||||
StreamProperties::Video(video) => assert_eq!(video.tmdb, Some(317981)),
|
||||
_ => panic!("expected video properties"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn merge_preserves_missing_series_tmdb_and_release_date() {
|
||||
let mut new_props = StreamProperties::Series(Box::new(SeriesStreamProperties {
|
||||
tmdb: None,
|
||||
release_date: None,
|
||||
..SeriesStreamProperties::default()
|
||||
}));
|
||||
let old_props = StreamProperties::Series(Box::new(SeriesStreamProperties {
|
||||
tmdb: Some(12345),
|
||||
release_date: Some("2015-01-01".into()),
|
||||
..SeriesStreamProperties::default()
|
||||
}));
|
||||
|
||||
let changed = merge_preserved_stream_properties(&mut new_props, &old_props);
|
||||
assert!(changed);
|
||||
match new_props {
|
||||
StreamProperties::Series(series) => {
|
||||
assert_eq!(series.tmdb, Some(12345));
|
||||
assert_eq!(series.release_date.as_deref(), Some("2015-01-01"));
|
||||
}
|
||||
_ => panic!("expected series properties"),
|
||||
}
|
||||
}
|
||||
|
||||
fn make_live_item(
|
||||
provider_id: u32,
|
||||
video: Option<&str>,
|
||||
|
||||
@@ -22,7 +22,6 @@ use shared::model::{
|
||||
MsgKind, PatternTemplate, SourcesConfigDto, TargetUserDto, TemplateDefinitionDto,
|
||||
};
|
||||
use shared::utils::{generate_default_access_secret, generate_default_encrypt_secret, CONSTANTS, PROVIDER_SCHEME_PREFIX, TEMPLATE_FILE};
|
||||
use std::borrow::Cow;
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::env;
|
||||
use std::fs::File;
|
||||
@@ -112,14 +111,19 @@ async fn parse_sources_file_from_path(
|
||||
.map_err(|join_err| info_err!("Failed to read sources-config file: {join_err}"))?
|
||||
}
|
||||
|
||||
pub fn resolve_template_and_mapping_paths<'a>(paths: &'a ConfigPaths, template_path: Option<&'a str>, mapping_path: Option<&'a str>) -> (Cow<'a, str>, Cow<'a, str>) {
|
||||
// Resolve effective paths for templates and mappings (with robust fallbacks)
|
||||
let effective_template_path = paths.template_file_path.as_deref()
|
||||
.or(template_path).map_or_else(|| Cow::Owned(utils::get_default_templates_path(&paths.config_path)), Cow::Borrowed);
|
||||
|
||||
let effective_mapping_path = paths.mapping_file_path.as_deref()
|
||||
.or(mapping_path).map_or_else(|| Cow::Owned(utils::get_default_mappings_path(&paths.config_path)), Cow::Borrowed);
|
||||
|
||||
pub fn resolve_template_and_mapping_paths(
|
||||
paths: &ConfigPaths,
|
||||
template_path: Option<&str>,
|
||||
mapping_path: Option<&str>,
|
||||
) -> (String, String) {
|
||||
let effective_template_path = utils::resolve_template_file_path(
|
||||
paths.config_path.as_str(),
|
||||
paths.template_file_path.as_deref().or(template_path),
|
||||
);
|
||||
let effective_mapping_path = utils::resolve_mapping_file_path(
|
||||
paths.config_path.as_str(),
|
||||
paths.mapping_file_path.as_deref().or(mapping_path),
|
||||
);
|
||||
(effective_template_path, effective_mapping_path)
|
||||
}
|
||||
|
||||
@@ -322,14 +326,15 @@ pub async fn read_app_config_dto(
|
||||
let config = read_config_file(config_file, resolve_env, include_computed)?;
|
||||
|
||||
// Resolve effective paths for templates and mappings (with robust fallbacks)
|
||||
let (effective_template_path,effective_mapping_path) = resolve_template_and_mapping_paths(paths, config.template_path.as_deref(), config.mapping_path.as_deref());
|
||||
let (effective_template_path, effective_mapping_path) =
|
||||
resolve_template_and_mapping_paths(paths, config.template_path.as_deref(), config.mapping_path.as_deref());
|
||||
|
||||
let mut sources = parse_sources_file_from_path(&PathBuf::from(sources_file), resolve_env).await?;
|
||||
let mut mappings = read_mappings_file_unprepared(effective_mapping_path.as_ref(), resolve_env)?
|
||||
let mut mappings = read_mappings_file_unprepared(&effective_mapping_path, resolve_env)?
|
||||
.map(|(_, mapping)| mapping);
|
||||
|
||||
let template_bundle = read_templates(
|
||||
Some(effective_template_path.as_ref()),
|
||||
Some(&effective_template_path),
|
||||
resolve_env,
|
||||
sources.templates.as_deref(),
|
||||
mappings
|
||||
@@ -447,7 +452,7 @@ pub async fn get_batch_aliases(
|
||||
if url.starts_with(PROVIDER_SCHEME_PREFIX) {
|
||||
return info_err_res!(
|
||||
"Batch input type '{input_type}' does not support provider:// URLs. \
|
||||
Use a local CSV path (absolute/relative) or file:// URL."
|
||||
Use a batch:// URL or a local CSV path (absolute/relative)."
|
||||
);
|
||||
}
|
||||
|
||||
@@ -505,26 +510,25 @@ pub async fn read_initial_app_config(
|
||||
|
||||
let config_dto = read_config_file(config_file, resolve_env, include_computed)?;
|
||||
|
||||
if paths.mapping_file_path.is_none() {
|
||||
let mut path = config_dto.mapping_path.as_ref().map_or_else(
|
||||
|| utils::get_default_mappings_path(config_path),
|
||||
ToString::to_string,
|
||||
);
|
||||
if resolve_env {
|
||||
path = resolve_env_var(&path);
|
||||
}
|
||||
paths.mapping_file_path.replace(path);
|
||||
}
|
||||
if paths.template_file_path.is_none() {
|
||||
let mut path = config_dto.template_path.as_ref().map_or_else(
|
||||
|| utils::get_default_templates_path(config_path),
|
||||
ToString::to_string,
|
||||
);
|
||||
if resolve_env {
|
||||
path = resolve_env_var(&path);
|
||||
}
|
||||
paths.template_file_path.replace(path);
|
||||
}
|
||||
let configured_mapping_path = paths
|
||||
.mapping_file_path
|
||||
.as_deref()
|
||||
.or(config_dto.mapping_path.as_deref())
|
||||
.map(|path| if resolve_env { resolve_env_var(path) } else { path.to_string() });
|
||||
let configured_template_path = paths
|
||||
.template_file_path
|
||||
.as_deref()
|
||||
.or(config_dto.template_path.as_deref())
|
||||
.map(|path| if resolve_env { resolve_env_var(path) } else { path.to_string() });
|
||||
|
||||
paths.mapping_file_path = Some(utils::resolve_mapping_file_path(
|
||||
config_path,
|
||||
configured_mapping_path.as_deref(),
|
||||
));
|
||||
paths.template_file_path = Some(utils::resolve_template_file_path(
|
||||
config_path,
|
||||
configured_template_path.as_deref(),
|
||||
));
|
||||
|
||||
let mut sources_dto = parse_sources_file_from_path(&PathBuf::from(sources_file), resolve_env).await?;
|
||||
|
||||
@@ -539,7 +543,7 @@ pub async fn read_initial_app_config(
|
||||
};
|
||||
|
||||
let template_bundle = read_templates(
|
||||
paths.template_file_path.as_deref().or(config_dto.template_path.as_deref()),
|
||||
paths.template_file_path.as_deref(),
|
||||
resolve_env,
|
||||
sources_dto.templates.as_deref(),
|
||||
mappings_dto
|
||||
@@ -562,7 +566,8 @@ pub async fn read_initial_app_config(
|
||||
|
||||
let sources: SourcesConfig = SourcesConfig::try_from(sources_dto)?;
|
||||
let mut config: Config = Config::from(config_dto);
|
||||
config.prepare(config_path)?;
|
||||
config.prepare(config_path, paths.home_path.as_str())?;
|
||||
paths.storage_path.clone_from(&config.storage_dir);
|
||||
config.update_runtime();
|
||||
|
||||
let mut app_config = AppConfig {
|
||||
|
||||
@@ -67,17 +67,27 @@ pub fn get_home_path() -> PathBuf {
|
||||
}
|
||||
|
||||
pub fn get_default_web_root_path() -> PathBuf {
|
||||
if let Some(web_root) = env::var(DEFAULT_WEB_ROOT_ENV_VAR)
|
||||
.ok()
|
||||
.filter(|p| !p.trim().is_empty()) {
|
||||
return PathBuf::from(web_root)
|
||||
get_default_web_root_path_for_home(get_home_path().as_path())
|
||||
}
|
||||
|
||||
pub fn get_default_web_root_path_for_home(home_path: &Path) -> PathBuf {
|
||||
if let Some(web_root) = env::var(DEFAULT_WEB_ROOT_ENV_VAR).ok().filter(|p| !p.trim().is_empty()) {
|
||||
let configured = PathBuf::from(web_root);
|
||||
return if configured.is_absolute() {
|
||||
configured
|
||||
} else {
|
||||
home_path.join(configured)
|
||||
};
|
||||
}
|
||||
get_exe_path().join(DEFAULT_WEB_DIR)
|
||||
get_default_path_for_home(home_path, DEFAULT_WEB_DIR)
|
||||
}
|
||||
|
||||
|
||||
pub fn get_default_path(file: &str) -> PathBuf {
|
||||
let home_path = get_home_path();
|
||||
get_default_path_for_home(get_home_path().as_path(), file)
|
||||
}
|
||||
|
||||
pub fn get_default_path_for_home(home_path: &Path, file: &str) -> PathBuf {
|
||||
home_path.join(file)
|
||||
}
|
||||
|
||||
@@ -116,11 +126,52 @@ pub fn get_default_templates_path(config_path: &str) -> String {
|
||||
get_default_file_path(config_path, TEMPLATE_FILE)
|
||||
}
|
||||
|
||||
fn resolve_config_scoped_path(config_path: &str, candidate: &str) -> String {
|
||||
let candidate_path = PathBuf::from(candidate.trim()).clean();
|
||||
if candidate_path.is_absolute() {
|
||||
return candidate_path.to_string_lossy().to_string();
|
||||
}
|
||||
|
||||
let config_dir = PathBuf::from(config_path).clean();
|
||||
let relative = if let Some(config_dir_name) = config_dir.file_name() {
|
||||
candidate_path
|
||||
.strip_prefix(Path::new(config_dir_name))
|
||||
.map_or_else(|_| candidate_path.clone(), Path::to_path_buf)
|
||||
} else {
|
||||
candidate_path.clone()
|
||||
};
|
||||
|
||||
config_dir.join(relative).clean().to_string_lossy().to_string()
|
||||
}
|
||||
|
||||
#[inline]
|
||||
pub fn resolve_mapping_file_path(config_path: &str, mapping_path: Option<&str>) -> String {
|
||||
let configured = mapping_path.map(str::trim).filter(|path| !path.is_empty()).map(ToString::to_string);
|
||||
let candidate = if shared::utils::is_blank_or_default_mapping_path(&configured) {
|
||||
get_default_mappings_path(config_path)
|
||||
} else {
|
||||
configured.unwrap_or_else(|| get_default_mappings_path(config_path))
|
||||
};
|
||||
resolve_config_scoped_path(config_path, &candidate)
|
||||
}
|
||||
|
||||
#[inline]
|
||||
pub fn resolve_template_file_path(config_path: &str, template_path: Option<&str>) -> String {
|
||||
let configured = template_path.map(str::trim).filter(|path| !path.is_empty()).map(ToString::to_string);
|
||||
let candidate = if shared::utils::is_blank_or_default_template_path(&configured) {
|
||||
get_default_templates_path(config_path)
|
||||
} else {
|
||||
configured.unwrap_or_else(|| get_default_templates_path(config_path))
|
||||
};
|
||||
resolve_config_scoped_path(config_path, &candidate)
|
||||
}
|
||||
|
||||
#[inline]
|
||||
pub fn resolve_template_persist_file_path(template_path: Option<&str>, config_path: &str) -> String {
|
||||
let candidate = template_path
|
||||
let raw_candidate = template_path
|
||||
.filter(|path| !path.trim().is_empty())
|
||||
.map_or_else(|| get_default_templates_path(config_path), ToString::to_string);
|
||||
let candidate = resolve_template_file_path(config_path, Some(raw_candidate.as_str()));
|
||||
let path = PathBuf::from(&candidate);
|
||||
|
||||
if path.exists() {
|
||||
@@ -130,8 +181,8 @@ pub fn resolve_template_persist_file_path(template_path: Option<&str>, config_pa
|
||||
return path.to_string_lossy().to_string();
|
||||
}
|
||||
|
||||
let looks_like_directory = candidate.ends_with('/')
|
||||
|| candidate.ends_with('\\')
|
||||
let looks_like_directory = raw_candidate.ends_with('/')
|
||||
|| raw_candidate.ends_with('\\')
|
||||
|| path.extension().is_none()
|
||||
|| path.extension().is_some_and(|ext| ext == "d");
|
||||
|
||||
@@ -329,11 +380,11 @@ where
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn prepare_file_path(persist: Option<&str>, working_dir: &str, action: &str) -> Option<PathBuf> {
|
||||
pub fn prepare_file_path(persist: Option<&str>, storage_dir: &str, action: &str) -> Option<PathBuf> {
|
||||
let persist_file: Option<PathBuf> =
|
||||
persist.map(|persist_path| prepare_persist_path(persist_path, action));
|
||||
if persist_file.is_some() {
|
||||
let file_path = get_file_path(working_dir, persist_file);
|
||||
let file_path = get_file_path(storage_dir, persist_file);
|
||||
debug_if_enabled!("persist to file: {}", file_path.as_ref().map_or(Cow::from("?"), |p| p.to_string_lossy()));
|
||||
file_path
|
||||
} else {
|
||||
@@ -348,27 +399,28 @@ pub fn read_file_as_bytes(path: &Path) -> std::io::Result<Vec<u8>> {
|
||||
Ok(buffer)
|
||||
}
|
||||
|
||||
pub fn make_absolute_path(path: &str, working_dir: &str) -> String {
|
||||
pub fn make_absolute_path(path: &str, storage_dir: &str) -> String {
|
||||
let rpb = std::path::PathBuf::from(path);
|
||||
let pathbuf = make_path_absolute(&rpb, working_dir);
|
||||
let pathbuf = make_path_absolute(&rpb, storage_dir);
|
||||
pathbuf.to_str().unwrap_or_default().to_string()
|
||||
}
|
||||
|
||||
pub fn make_path_absolute(rpb: &Path, working_dir: &str) -> PathBuf {
|
||||
pub fn make_path_absolute(rpb: &Path, storage_dir: &str) -> PathBuf {
|
||||
if rpb.is_relative() {
|
||||
let mut rpb2 = std::path::PathBuf::from(working_dir).join(rpb);
|
||||
if !rpb2.exists() {
|
||||
rpb2 = get_exe_path().join(rpb);
|
||||
if !storage_dir.trim().is_empty() {
|
||||
return std::path::PathBuf::from(storage_dir).join(rpb).clean();
|
||||
}
|
||||
if !rpb2.exists() {
|
||||
let cwd = std::env::current_dir();
|
||||
if let Ok(cwd_path) = cwd {
|
||||
rpb2 = cwd_path.join(rpb);
|
||||
}
|
||||
|
||||
let exe_based = get_exe_path().join(rpb).clean();
|
||||
if exe_based.is_absolute() {
|
||||
return exe_based;
|
||||
}
|
||||
if rpb2.exists() {
|
||||
return rpb2.clean();
|
||||
|
||||
if let Ok(cwd_path) = std::env::current_dir() {
|
||||
return cwd_path.join(rpb).clean();
|
||||
}
|
||||
|
||||
return exe_based;
|
||||
}
|
||||
rpb.to_path_buf()
|
||||
}
|
||||
@@ -473,7 +525,11 @@ pub fn get_file_extension(path: &str) -> Option<String> {
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::normalize_string_path;
|
||||
use super::{
|
||||
normalize_string_path, resolve_mapping_file_path, resolve_template_file_path,
|
||||
resolve_template_persist_file_path,
|
||||
};
|
||||
use std::path::PathBuf;
|
||||
|
||||
#[test]
|
||||
fn test_simple_relative_path() {
|
||||
@@ -509,4 +565,40 @@ mod tests {
|
||||
let normalized = normalize_string_path(input);
|
||||
assert_eq!(normalized, "");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_resolve_template_default_path_uses_config_dir() {
|
||||
let config_path = std::env::temp_dir().join("tuliprox").join("settings").join("config");
|
||||
let resolved = resolve_template_file_path(
|
||||
config_path.to_string_lossy().as_ref(),
|
||||
Some("./config/template.yml"),
|
||||
);
|
||||
let expected = config_path.join("template.yml");
|
||||
assert_eq!(PathBuf::from(resolved), expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_resolve_template_directory_path_uses_config_dir() {
|
||||
let config_path = std::env::temp_dir().join("tuliprox").join("settings").join("config");
|
||||
let resolved = resolve_template_file_path(config_path.to_string_lossy().as_ref(), Some("template.d"));
|
||||
let expected = config_path.join("template.d");
|
||||
assert_eq!(PathBuf::from(resolved), expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_resolve_mapping_directory_path_uses_config_dir() {
|
||||
let config_path = std::env::temp_dir().join("tuliprox").join("settings").join("config");
|
||||
let resolved = resolve_mapping_file_path(config_path.to_string_lossy().as_ref(), Some("mapping.d"));
|
||||
let expected = config_path.join("mapping.d");
|
||||
assert_eq!(PathBuf::from(resolved), expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_resolve_template_persist_file_for_directory_path() {
|
||||
let config_path = std::env::temp_dir().join("tuliprox").join("settings").join("config");
|
||||
let resolved =
|
||||
resolve_template_persist_file_path(Some("template.d"), config_path.to_string_lossy().as_ref());
|
||||
let expected = config_path.join("template.d").join("template.yml");
|
||||
assert_eq!(PathBuf::from(resolved), expected);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,28 +1,47 @@
|
||||
use crate::utils::traverse_dir;
|
||||
use crate::utils::{config_file_reader, open_file};
|
||||
use log::warn;
|
||||
use log::{debug, warn};
|
||||
use shared::error::{info_err_res, TuliproxError};
|
||||
use shared::info_err;
|
||||
use shared::model::TemplateDefinitionDto;
|
||||
use std::io::ErrorKind;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
fn read_template_definition(
|
||||
template_file: &Path,
|
||||
resolve_var: bool,
|
||||
) -> Result<Option<TemplateDefinitionDto>, TuliproxError> {
|
||||
if let Ok(file) = open_file(template_file) {
|
||||
let maybe_definition: Result<TemplateDefinitionDto, _> =
|
||||
serde_saphyr::from_reader(config_file_reader(file, resolve_var));
|
||||
return match maybe_definition {
|
||||
Ok(definition) => Ok(Some(definition)),
|
||||
Err(err) => info_err_res!("{err}"),
|
||||
};
|
||||
match open_file(template_file) {
|
||||
Ok(file) => {
|
||||
let maybe_definition: Result<TemplateDefinitionDto, _> =
|
||||
serde_saphyr::from_reader(config_file_reader(file, resolve_var));
|
||||
match maybe_definition {
|
||||
Ok(definition) => Ok(Some(definition)),
|
||||
Err(err) => info_err_res!(
|
||||
"Failed to parse template file {}: {err}",
|
||||
template_file.to_string_lossy()
|
||||
),
|
||||
}
|
||||
}
|
||||
Err(err) => {
|
||||
if err.kind() == ErrorKind::NotFound {
|
||||
debug!(
|
||||
"Optional template file not found: {}",
|
||||
template_file.to_str().unwrap_or("?")
|
||||
);
|
||||
Ok(None)
|
||||
} else {
|
||||
warn!(
|
||||
"Can't read template file {}: {err}",
|
||||
template_file.to_str().unwrap_or("?")
|
||||
);
|
||||
info_err_res!(
|
||||
"Can't read template file {}: {err}",
|
||||
template_file.to_string_lossy()
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
warn!(
|
||||
"Can't read template file: {}",
|
||||
template_file.to_str().unwrap_or("?")
|
||||
);
|
||||
Ok(None)
|
||||
}
|
||||
|
||||
fn merge_template_definitions(definitions: Vec<TemplateDefinitionDto>) -> Option<TemplateDefinitionDto> {
|
||||
@@ -104,11 +123,19 @@ pub fn read_templates_file(
|
||||
}
|
||||
}
|
||||
Err(err) => {
|
||||
warn!(
|
||||
"Can't read template path metadata for {}: {err}",
|
||||
path.to_string_lossy()
|
||||
);
|
||||
Ok(None)
|
||||
if err.kind() == ErrorKind::NotFound {
|
||||
debug!("Optional template path not found: {}", path.to_string_lossy());
|
||||
Ok(None)
|
||||
} else {
|
||||
warn!(
|
||||
"Can't read template path metadata for {}: {err}",
|
||||
path.to_string_lossy()
|
||||
);
|
||||
info_err_res!(
|
||||
"Can't read template path metadata for {}: {err}",
|
||||
path.to_string_lossy()
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -11,28 +11,28 @@ use shared::error::{info_err, TuliproxError};
|
||||
use shared::utils::{sanitize_sensitive_info, short_hash};
|
||||
use std::path::PathBuf;
|
||||
|
||||
pub async fn get_input_raw_epg_file_path(url: &str, input: &ConfigInput, working_dir: &str) -> std::io::Result<PathBuf> {
|
||||
pub async fn get_input_raw_epg_file_path(url: &str, input: &ConfigInput, storage_dir: &str) -> std::io::Result<PathBuf> {
|
||||
let file_prefix = short_hash(url);
|
||||
|
||||
if let Some(persist_path) = input.persist.as_deref() {
|
||||
if !persist_path.is_empty() {
|
||||
if let Some(path) = prepare_file_path(input.persist.as_deref(), working_dir, "")
|
||||
if let Some(path) = prepare_file_path(input.persist.as_deref(), storage_dir, "")
|
||||
.map(|path| add_prefix_to_filename(&path, concat_string!(&file_prefix, "_epg_").as_str(), Some("xml"))) {
|
||||
return Ok(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let download_path = get_input_storage_path(&input.name, working_dir).await?;
|
||||
let download_path = get_input_storage_path(&input.name, storage_dir).await?;
|
||||
Ok(download_path.join(format!("{}_{}", file_prefix, storage_const::FILE_EPG)))
|
||||
}
|
||||
|
||||
async fn download_epg_file(url: &str, ctx: &PlaylistProcessingContext,
|
||||
input: &ConfigInput,
|
||||
headers: Option<&reqwest::header::HeaderMap>,
|
||||
working_dir: &str) -> Result<PathBuf, TuliproxError> {
|
||||
storage_dir: &str) -> Result<PathBuf, TuliproxError> {
|
||||
debug!("Getting epg file path for url: {}", sanitize_sensitive_info(url));
|
||||
let persist_file_path = get_input_raw_epg_file_path(url, input, working_dir).await.map_err(|e| info_err!("Could not access epg file download directory: {}", e))?;
|
||||
let persist_file_path = get_input_raw_epg_file_path(url, input, storage_dir).await.map_err(|e| info_err!("Could not access epg file download directory: {}", e))?;
|
||||
|
||||
if input.cache_duration_seconds > 0 {
|
||||
if let Ok(metadata) = tokio::fs::metadata(&persist_file_path).await {
|
||||
@@ -55,7 +55,7 @@ async fn download_epg_file(url: &str, ctx: &PlaylistProcessingContext,
|
||||
return Ok(persist_file_path);
|
||||
}
|
||||
debug!("Downloading epg for input '{}'", input.name);
|
||||
match request::get_input_epg_content_as_file(&ctx.config, &ctx.client, input, headers, working_dir, url, &persist_file_path).await {
|
||||
match request::get_input_epg_content_as_file(&ctx.config, &ctx.client, input, headers, storage_dir, url, &persist_file_path).await {
|
||||
Ok(path) => {
|
||||
ctx.mark_input_downloaded(lock_key.clone()).await;
|
||||
Ok(path)
|
||||
@@ -66,7 +66,7 @@ async fn download_epg_file(url: &str, ctx: &PlaylistProcessingContext,
|
||||
|
||||
pub async fn get_xmltv(ctx: &PlaylistProcessingContext, input: &ConfigInput,
|
||||
headers: Option<&reqwest::header::HeaderMap>,
|
||||
working_dir: &str) -> (Option<TVGuide>, Vec<TuliproxError>) {
|
||||
storage_dir: &str) -> (Option<TVGuide>, Vec<TuliproxError>) {
|
||||
match &input.epg {
|
||||
None => (None, vec![]),
|
||||
Some(epg_config) => {
|
||||
@@ -75,7 +75,7 @@ pub async fn get_xmltv(ctx: &PlaylistProcessingContext, input: &ConfigInput,
|
||||
let mut stored_file_paths = vec![];
|
||||
|
||||
for epg_source in &epg_config.sources {
|
||||
match download_epg_file(&epg_source.url, ctx, input, headers, working_dir).await {
|
||||
match download_epg_file(&epg_source.url, ctx, input, headers, storage_dir).await {
|
||||
Ok(file_path) => {
|
||||
stored_file_paths.push(file_path.clone());
|
||||
file_paths.push(PersistedEpgSource { file_path, priority: epg_source.priority, logo_override: epg_source.logo_override });
|
||||
|
||||
@@ -12,19 +12,19 @@ pub async fn download_m3u_playlist(
|
||||
cfg: &Arc<Config>,
|
||||
input: &ConfigInput,
|
||||
) -> (Vec<PlaylistGroup>, Vec<TuliproxError>) {
|
||||
let working_dir = &cfg.working_dir;
|
||||
let storage_dir = &cfg.storage_dir;
|
||||
let input_source: InputSource = {
|
||||
match input.staged.as_ref() {
|
||||
None => input.into(),
|
||||
Some(staged) => if staged.enabled { staged.into() } else { input.into() },
|
||||
}
|
||||
};
|
||||
let persist_file_path = prepare_file_path(input.persist.as_deref(), working_dir, "");
|
||||
let persist_file_path = prepare_file_path(input.persist.as_deref(), storage_dir, "");
|
||||
match request::get_input_text_content_as_stream(
|
||||
app_config,
|
||||
client,
|
||||
&input_source,
|
||||
working_dir,
|
||||
storage_dir,
|
||||
persist_file_path,
|
||||
)
|
||||
.await
|
||||
|
||||
@@ -20,7 +20,7 @@ use reqwest::{
|
||||
};
|
||||
use shared::{
|
||||
error::{notify_err_res, string_to_io_error, TuliproxError},
|
||||
model::{format_elapsed_time, InputFetchMethod, OnConnectErrorPolicy, DEFAULT_USER_AGENT},
|
||||
model::{format_elapsed_time, InputFetchMethod, OnConnectErrorPolicy},
|
||||
utils::{
|
||||
filter_request_header, human_readable_byte_size, sanitize_sensitive_info, CONTENT_TYPE_JSON, ENCODING_DEFLATE,
|
||||
ENCODING_GZIP,
|
||||
@@ -42,6 +42,7 @@ use tokio::{
|
||||
};
|
||||
use tokio_util::io::StreamReader;
|
||||
use url::Url;
|
||||
use shared::utils::DEFAULT_USER_AGENT;
|
||||
|
||||
static PROXY_DIAGNOSTICS_ONCE: Once = Once::new();
|
||||
|
||||
@@ -533,13 +534,13 @@ pub async fn get_input_epg_content_as_file(
|
||||
client: &reqwest::Client,
|
||||
input: &ConfigInput,
|
||||
headers: Option<&HeaderMap>,
|
||||
working_dir: &str,
|
||||
storage_dir: &str,
|
||||
url_str: &str,
|
||||
persist_filepath: &Path,
|
||||
) -> Result<PathBuf, TuliproxError> {
|
||||
debug_if_enabled!(
|
||||
"getting input epg content working_dir: {}, url: {}",
|
||||
working_dir,
|
||||
"getting input epg content storage_dir: {}, url: {}",
|
||||
storage_dir,
|
||||
sanitize_sensitive_info(url_str)
|
||||
);
|
||||
if url_str.parse::<url::Url>().is_ok() {
|
||||
@@ -556,7 +557,7 @@ pub async fn get_input_epg_content_as_file(
|
||||
}
|
||||
}
|
||||
} else {
|
||||
let result = match get_file_path(working_dir, Some(PathBuf::from(url_str))) {
|
||||
let result = match get_file_path(storage_dir, Some(PathBuf::from(url_str))) {
|
||||
Some(filepath) => {
|
||||
if filepath.exists() {
|
||||
if let Err(e) = tokio::fs::copy(&filepath, persist_filepath).await {
|
||||
@@ -590,12 +591,12 @@ pub async fn get_input_text_content(
|
||||
app_state: &Arc<AppState>,
|
||||
client: &reqwest::Client,
|
||||
input: &InputSource,
|
||||
working_dir: &str,
|
||||
storage_dir: &str,
|
||||
persist_filepath: Option<PathBuf>,
|
||||
) -> Result<String, TuliproxError> {
|
||||
debug_if_enabled!(
|
||||
"getting input text content working_dir: {}, url: {}",
|
||||
working_dir,
|
||||
"getting input text content storage_dir: {}, url: {}",
|
||||
storage_dir,
|
||||
sanitize_sensitive_info(&input.url)
|
||||
);
|
||||
|
||||
@@ -612,7 +613,7 @@ pub async fn get_input_text_content(
|
||||
}
|
||||
}
|
||||
} else {
|
||||
let result = match get_file_path(working_dir, Some(PathBuf::from(&input.url))) {
|
||||
let result = match get_file_path(storage_dir, Some(PathBuf::from(&input.url))) {
|
||||
Some(filepath) => {
|
||||
if filepath.exists() {
|
||||
if let Some(persist_file_value) = persist_filepath {
|
||||
@@ -650,12 +651,12 @@ pub async fn get_input_text_content_as_stream(
|
||||
app_config: &Arc<AppConfig>,
|
||||
client: &reqwest::Client,
|
||||
input: &InputSource,
|
||||
working_dir: &str,
|
||||
storage_dir: &str,
|
||||
persist_filepath: Option<PathBuf>,
|
||||
) -> Result<DynReader, TuliproxError> {
|
||||
debug_if_enabled!(
|
||||
"getting input text content working_dir: {}, url: {}",
|
||||
working_dir,
|
||||
"getting input text content storage_dir: {}, url: {}",
|
||||
storage_dir,
|
||||
sanitize_sensitive_info(&input.url)
|
||||
);
|
||||
|
||||
@@ -672,7 +673,7 @@ pub async fn get_input_text_content_as_stream(
|
||||
}
|
||||
}
|
||||
} else {
|
||||
let result = match get_file_path(working_dir, Some(PathBuf::from(&input.url))) {
|
||||
let result = match get_file_path(storage_dir, Some(PathBuf::from(&input.url))) {
|
||||
Some(filepath) => {
|
||||
if filepath.exists() {
|
||||
match get_local_file_content_as_stream(&filepath).await {
|
||||
@@ -1506,7 +1507,7 @@ mod tests {
|
||||
};
|
||||
use crate::{
|
||||
model::{AppConfig, Config, ConfigProvider, ResourceRetryConfig, ReverseProxyConfig, SourcesConfig},
|
||||
utils::FileLockManager,
|
||||
utils::{FileLockManager, DEFAULT_USER_AGENT}
|
||||
};
|
||||
use arc_swap::{ArcSwap, ArcSwapOption};
|
||||
use shared::model::{
|
||||
@@ -1536,7 +1537,9 @@ mod tests {
|
||||
api_proxy: Arc::new(ArcSwapOption::default()),
|
||||
file_locks: Arc::new(FileLockManager::default()),
|
||||
paths: Arc::new(ArcSwap::from_pointee(ConfigPaths {
|
||||
home_path: String::new(),
|
||||
config_path: String::new(),
|
||||
storage_path: String::new(),
|
||||
config_file_path: String::new(),
|
||||
sources_file_path: String::new(),
|
||||
mapping_file_path: None,
|
||||
@@ -1631,7 +1634,7 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn test_get_request_headers_prioritization() {
|
||||
use super::{get_request_headers, DEFAULT_USER_AGENT};
|
||||
use super::{get_request_headers};
|
||||
use axum::http::header::USER_AGENT;
|
||||
|
||||
// Case 1: No headers provided -> Default UA
|
||||
|
||||
@@ -14,7 +14,7 @@ use chrono::{DateTime, Utc};
|
||||
use log::{error, info, warn};
|
||||
use shared::error::TuliproxError;
|
||||
use shared::model::{PlaylistEntry, PlaylistGroup, ProxyUserStatus, SeriesStreamProperties,
|
||||
StreamProperties, VideoStreamProperties, XtreamCluster, XtreamPlaylistItem,
|
||||
StreamProperties, VideoStreamProperties, InputType, XtreamCluster, XtreamPlaylistItem,
|
||||
XtreamSeriesInfo, XtreamVideoInfo, XtreamVideoInfoDoc};
|
||||
use shared::utils::{extract_extension_from_url, get_i64_from_serde_value, get_string_from_serde_value, sanitize_sensitive_info, Internable, PROVIDER_SCHEME_PREFIX};
|
||||
use std::collections::HashMap;
|
||||
@@ -93,8 +93,8 @@ pub async fn get_xtream_stream_info(client: &reqwest::Client,
|
||||
match cluster {
|
||||
XtreamCluster::Live => {}
|
||||
XtreamCluster::Video => {
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
if let Ok(storage_path) = get_input_storage_path(&input.name, working_dir).await {
|
||||
let storage_dir = &app_config.config.load().storage_dir;
|
||||
if let Ok(storage_path) = get_input_storage_path(&input.name, storage_dir).await {
|
||||
match serde_json::from_str::<XtreamVideoInfo>(&content) {
|
||||
Ok(info) => {
|
||||
// parse downloaded info into StreamProperties
|
||||
@@ -126,7 +126,7 @@ pub async fn get_xtream_stream_info(client: &reqwest::Client,
|
||||
}
|
||||
}
|
||||
XtreamCluster::Series => {
|
||||
let working_dir = &app_config.config.load().working_dir;
|
||||
let storage_dir = &app_config.config.load().storage_dir;
|
||||
let group = pli.get_group();
|
||||
let series_name = pli.get_name();
|
||||
|
||||
@@ -135,7 +135,7 @@ pub async fn get_xtream_stream_info(client: &reqwest::Client,
|
||||
// parse series info
|
||||
let series_stream_props = SeriesStreamProperties::from_info(&info, pli);
|
||||
|
||||
if let Ok(storage_path) = get_input_storage_path(&input.name, working_dir).await {
|
||||
if let Ok(storage_path) = get_input_storage_path(&input.name, storage_dir).await {
|
||||
// update input db
|
||||
if let Err(err) = persists_input_series_info(app_config, &storage_path, cluster, &input.name, provider_id, &series_stream_props).await {
|
||||
error!("Failed to persist series info for input {}: {err}", &input.name);
|
||||
@@ -387,7 +387,11 @@ pub async fn download_xtream_playlist(app_config: &Arc<AppConfig>, client: &reqw
|
||||
let skip_cluster = get_skip_cluster(input);
|
||||
|
||||
let cfg = app_config.config.load();
|
||||
let working_dir = &cfg.working_dir;
|
||||
let storage_dir = &cfg.storage_dir;
|
||||
// Staged input type is the effective type during download.
|
||||
let effective_type = input.get_download_input_type();
|
||||
let use_disk_based_processing =
|
||||
cfg.disk_based_processing && matches!(effective_type, InputType::Xtream | InputType::XtreamBatch);
|
||||
|
||||
let mut errors = vec![];
|
||||
for (xtream_cluster, category, stream) in &ACTIONS {
|
||||
@@ -396,16 +400,16 @@ pub async fn download_xtream_playlist(app_config: &Arc<AppConfig>, client: &reqw
|
||||
let input_source_category = input_source.with_url(concat_string!(&base_url, "&action=", category));
|
||||
let input_source_stream = input_source.with_url(concat_string!(&base_url, "&action=", stream));
|
||||
let category_file_path = crate::utils::prepare_file_path(input.persist.as_deref(),
|
||||
working_dir, concat_string!(category, "_").as_str());
|
||||
storage_dir, concat_string!(category, "_").as_str());
|
||||
let stream_file_path = crate::utils::prepare_file_path(input.persist.as_deref(),
|
||||
working_dir, concat_string!(stream, "_").as_str());
|
||||
storage_dir, concat_string!(stream, "_").as_str());
|
||||
|
||||
match futures::join!(
|
||||
request::get_input_json_content_as_stream(app_config, client, &input_source_category, category_file_path),
|
||||
request::get_input_json_content_as_stream(app_config, client, &input_source_stream, stream_file_path)
|
||||
) {
|
||||
(Ok(category_content), Ok(stream_content)) => {
|
||||
if cfg.disk_based_processing {
|
||||
if use_disk_based_processing {
|
||||
// trace!("Using disk input playlist optimization for cluster {}", xtream_cluster);
|
||||
if let Err(err) = persist_input_xtream_playlist_cluster_to_disk(app_config, input, *xtream_cluster, category_content, stream_content).await {
|
||||
error!("persist_input_xtream_playlist_cluster_to_disk failed: {err}");
|
||||
@@ -443,7 +447,7 @@ pub async fn download_xtream_playlist(app_config: &Arc<AppConfig>, client: &reqw
|
||||
plg.id = grp_id;
|
||||
}
|
||||
|
||||
(playlist_groups, errors, cfg.disk_based_processing)
|
||||
(playlist_groups, errors, use_disk_based_processing)
|
||||
}
|
||||
|
||||
async fn check_alias_user_state(app_config: &Arc<AppConfig>, client: &reqwest::Client, input: &ConfigInput) {
|
||||
|
||||
@@ -3,8 +3,7 @@ use shared::error::TuliproxError;
|
||||
use reqwest::header::{HeaderMap, HeaderValue};
|
||||
use log::{debug, info};
|
||||
use shared::error::{info_err};
|
||||
use shared::model::DEFAULT_USER_AGENT;
|
||||
use shared::utils::{trim_last_slash, TRAKT_API_KEY};
|
||||
use shared::utils::{trim_last_slash, DEFAULT_USER_AGENT, TRAKT_API_KEY};
|
||||
use super::errors::{handle_trakt_api_error};
|
||||
|
||||
pub struct TraktClient {
|
||||
|
||||
@@ -286,7 +286,7 @@
|
||||
"NAME": "Human readable name for this entry. Must be unique.",
|
||||
"PASSWORD": "Password or credential used for this account or service.",
|
||||
"PRIORITY": "The order of processing. Lower numbers mean higher priority (e.g. -1 is higher than 0).",
|
||||
"URL": "Target URL for this input. Supports provider:// scheme for failover.",
|
||||
"URL": "Input source URL. Supported schemes:\n\n```http(s)://\nfile://\nprovider://<provider>/...\nbatch://...(auto-switches m3u/xtream to batch types).```\n\nFor m3u_batch/xtream_batch only local CSV sources are allowed (batch://...).",
|
||||
"USERNAME": "Login username for this account or service."
|
||||
},
|
||||
"CONFIG_INPUT_ALIAS": {
|
||||
@@ -425,7 +425,7 @@
|
||||
"UPDATE_ON_BOOT": "If enabled, Tuliprox triggers a full playlist update immediately upon startup. Helpful during initial setup, but usually set to false for stable operations.",
|
||||
"USER_ACCESS_CONTROL": "If enabled, the attributes (if set)\n\n- expiration date,\n- status and\n- max_connections\n\nare checked to permit or deny access for api users.",
|
||||
"USER_CONFIG_DIR": "Directory path where user-specific overrides and configuration files are stored.",
|
||||
"WORKING_DIR": "The primary directory where Tuliprox stores its data files and databases."
|
||||
"STORAGE_DIR": "The primary directory where Tuliprox stores its data files and databases."
|
||||
},
|
||||
"MESSAGING_CONFIG": {
|
||||
"NOTIFY_ON": "Select events (e.g. 'info', 'stats', 'error') that trigger notifications for this specific type.",
|
||||
@@ -544,7 +544,7 @@
|
||||
"STAGED_INPUT": {
|
||||
"HEADERS": "Custom HTTP headers sent when retrieving this staged configuration.",
|
||||
"PASSWORD": "Password or credential used for this account or service.",
|
||||
"URL": "Target URL for this setting.",
|
||||
"URL": "Source URL for the staged input. Use http(s)://, file:// or provider://<provider>/... . Batch URLs are not supported for staged inputs.",
|
||||
"USERNAME": "Login username for this account or service."
|
||||
},
|
||||
"STREAM_BUFFER_CONFIG": {
|
||||
@@ -1138,7 +1138,7 @@
|
||||
"WEB_UI_CONFIG": "Web-Ui",
|
||||
"WEB_UI_PATH": "Path prefix",
|
||||
"WELCOME": "Welcome",
|
||||
"WORKING_DIR": "Working dir",
|
||||
"STORAGE_DIR": "Storage dir",
|
||||
"XC": "xc",
|
||||
"XTREAM": "xtream",
|
||||
"XTREAM_BATCH": "xc batch",
|
||||
|
||||
@@ -21,7 +21,9 @@
|
||||
|
||||
&__list,
|
||||
&__text {
|
||||
display: flex;
|
||||
flex-flow: column;
|
||||
gap: var(--gap-default);
|
||||
}
|
||||
|
||||
&__value {
|
||||
@@ -40,6 +42,8 @@
|
||||
border-radius: var(--border-radius);
|
||||
box-sizing: border-box;
|
||||
white-space: nowrap;
|
||||
overflow-x: auto;
|
||||
scrollbar-width: none;
|
||||
}
|
||||
|
||||
&__bool {
|
||||
|
||||
@@ -137,6 +137,14 @@
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.tp__form-list {
|
||||
display: flex;
|
||||
flex-flow: column;
|
||||
gap: var(--gap-default);
|
||||
box-sizing: border-box;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.tp__form-list__item-arrow-btn {
|
||||
svg {
|
||||
transform: scale(2);
|
||||
|
||||
@@ -92,4 +92,8 @@ $form-width: 520px;
|
||||
.tp__source-editor-form.disabled {
|
||||
pointer-events: none;
|
||||
opacity: 0.3;
|
||||
}
|
||||
|
||||
.tp__input-form__body {
|
||||
width: 100%;
|
||||
}
|
||||
@@ -184,9 +184,13 @@ pub fn ApiConfigView() -> Html {
|
||||
let deps = (api_config, *config_view_ctx.edit_mode);
|
||||
use_effect_with(deps, move |(cfg, _mode)| {
|
||||
if let Some(api) = cfg {
|
||||
form_state_api_config.dispatch(ApiConfigFormAction::SetAll(api.clone()));
|
||||
let mut next_api = api.clone();
|
||||
next_api.prepare();
|
||||
form_state_api_config.dispatch(ApiConfigFormAction::SetAll(next_api));
|
||||
} else {
|
||||
form_state_api_config.dispatch(ApiConfigFormAction::SetAll(ConfigApiDto::default()));
|
||||
let mut defaults = ConfigApiDto::default();
|
||||
defaults.prepare();
|
||||
form_state_api_config.dispatch(ApiConfigFormAction::SetAll(defaults));
|
||||
}
|
||||
|| ()
|
||||
});
|
||||
@@ -421,11 +425,13 @@ pub fn ApiConfigView() -> Html {
|
||||
|
||||
let render_view_mode_api_config = || {
|
||||
if let Some(config) = &config_ctx.config {
|
||||
let mut api_config = config.config.api.clone();
|
||||
api_config.prepare();
|
||||
html! {
|
||||
<Card class="tp__api-config-card">
|
||||
{ config_field!(config.config.api, translate.t(LABEL_HOST), host) }
|
||||
{ config_field!(config.config.api, translate.t(LABEL_PORT), port) }
|
||||
{ config_field!(config.config.api, translate.t(LABEL_WEB_ROOT), web_root) }
|
||||
{ config_field!(api_config, translate.t(LABEL_HOST), host) }
|
||||
{ config_field!(api_config, translate.t(LABEL_PORT), port) }
|
||||
{ config_field!(api_config, translate.t(LABEL_WEB_ROOT), web_root) }
|
||||
</Card>
|
||||
}
|
||||
} else {
|
||||
|
||||
@@ -474,7 +474,7 @@ pub fn ConfigView() -> Html {
|
||||
|
||||
{ html_if!(*edit_mode || setup_mode, {
|
||||
<div class="tp__config-view__toolbar tp__form-page__toolbar">
|
||||
<TextButton class="primary" name="save_config"
|
||||
<TextButton class="secondary" name="save_config"
|
||||
icon="Save"
|
||||
title={ if setup_mode { translate.t(LABEL_SETUP_FINISH) } else { translate.t(LABEL_SAVE) }}
|
||||
onclick={handle_save_config}></TextButton>
|
||||
|
||||
@@ -7,10 +7,10 @@ use crate::{
|
||||
context::ConfigContext,
|
||||
},
|
||||
config_field, config_field_bool, config_field_optional, edit_field_bool, edit_field_number,
|
||||
edit_field_number_option_u32, edit_field_text, edit_field_text_option, generate_form_reducer,
|
||||
edit_field_number_option_u32, edit_field_text_option, generate_form_reducer,
|
||||
i18n::use_translation,
|
||||
};
|
||||
use shared::model::MainConfigDto;
|
||||
use shared::model::{ConfigDto, MainConfigDto};
|
||||
use yew::prelude::*;
|
||||
|
||||
const LABEL_UPDATE_ON_BOOT: &str = "LABEL.UPDATE_ON_BOOT";
|
||||
@@ -18,7 +18,7 @@ const LABEL_CONFIG_HOT_RELOAD: &str = "LABEL.CONFIG_HOT_RELOAD";
|
||||
const LABEL_USER_ACCESS_CONTROL: &str = "LABEL.USER_ACCESS_CONTROL";
|
||||
const LABEL_PROCESS_PARALLEL: &str = "LABEL.PROCESS_PARALLEL";
|
||||
const LABEL_DISK_BASED_PROCESSING: &str = "LABEL.DISK_BASED_PROCESSING";
|
||||
const LABEL_WORKING_DIR: &str = "LABEL.WORKING_DIR";
|
||||
const LABEL_STORAGE_DIR: &str = "LABEL.STORAGE_DIR";
|
||||
const LABEL_DEFAULT_USER_AGENT: &str = "LABEL.DEFAULT_USER_AGENT";
|
||||
const LABEL_MAPPING_PATH: &str = "LABEL.MAPPING_PATH";
|
||||
const LABEL_TEMPLATE_PATH: &str = "LABEL.TEMPLATE_PATH";
|
||||
@@ -39,7 +39,7 @@ generate_form_reducer!(
|
||||
AcceptInsecureSslCertificates => accept_insecure_ssl_certificates: bool,
|
||||
ProcessParallel => process_parallel: bool,
|
||||
DiskBasedProcessing => disk_based_processing: bool,
|
||||
WorkingDir => working_dir: String,
|
||||
StorageDir => storage_dir: Option<String>,
|
||||
DefaultUserAgent => default_user_agent: Option<String>,
|
||||
MappingPath => mapping_path: Option<String>,
|
||||
TemplatePath => template_path: Option<String>,
|
||||
@@ -73,8 +73,16 @@ pub fn MainConfigView() -> Html {
|
||||
let config = config_ctx.config.as_ref().map(|c| c.config.clone());
|
||||
use_effect_with((config, config_view_ctx.edit_mode.clone()), move |(cfg, _mode)| {
|
||||
if let Some(main) = cfg {
|
||||
let main_config = MainConfigDto::from(main);
|
||||
form_state.dispatch(MainConfigFormAction::SetAll(main_config.clone()));
|
||||
let mut prepared_main: ConfigDto = main.clone();
|
||||
match prepared_main.prepare(false) {
|
||||
Ok(()) => {
|
||||
let main_config = MainConfigDto::from(&prepared_main);
|
||||
form_state.dispatch(MainConfigFormAction::SetAll(main_config.clone()));
|
||||
}
|
||||
Err(err) => {
|
||||
log::error!("Failed to prepare main config for view state: {err}");
|
||||
}
|
||||
}
|
||||
} else {
|
||||
form_state.dispatch(MainConfigFormAction::SetAll(MainConfigDto::default()));
|
||||
}
|
||||
@@ -91,7 +99,7 @@ pub fn MainConfigView() -> Html {
|
||||
{ config_field_bool!(form_state.form, translate.t(LABEL_ACCEPT_INSECURE_SSL_CERTIFICATES), accept_insecure_ssl_certificates) }
|
||||
{ config_field_bool!(form_state.form, translate.t(LABEL_PROCESS_PARALLEL), process_parallel) }
|
||||
{ config_field_bool!(form_state.form, translate.t(LABEL_DISK_BASED_PROCESSING), disk_based_processing) }
|
||||
{ config_field!(form_state.form, translate.t(LABEL_WORKING_DIR), working_dir) }
|
||||
{ config_field_optional!(form_state.form, translate.t(LABEL_STORAGE_DIR), storage_dir) }
|
||||
{ config_field_optional!(form_state.form, translate.t(LABEL_DEFAULT_USER_AGENT), default_user_agent) }
|
||||
{ config_field_optional!(form_state.form, translate.t(LABEL_MAPPING_PATH), mapping_path) }
|
||||
{ config_field_optional!(form_state.form, translate.t(LABEL_TEMPLATE_PATH), template_path) }
|
||||
@@ -113,7 +121,7 @@ pub fn MainConfigView() -> Html {
|
||||
{ edit_field_bool!(form_state, translate.t(LABEL_ACCEPT_INSECURE_SSL_CERTIFICATES), accept_insecure_ssl_certificates, MainConfigFormAction::AcceptInsecureSslCertificates) }
|
||||
{ edit_field_bool!(form_state, translate.t(LABEL_PROCESS_PARALLEL), process_parallel, MainConfigFormAction::ProcessParallel) }
|
||||
{ edit_field_bool!(form_state, translate.t(LABEL_DISK_BASED_PROCESSING), disk_based_processing, MainConfigFormAction::DiskBasedProcessing) }
|
||||
{ edit_field_text!(form_state, translate.t(LABEL_WORKING_DIR), working_dir, MainConfigFormAction::WorkingDir) }
|
||||
{ edit_field_text_option!(form_state, translate.t(LABEL_STORAGE_DIR), storage_dir, MainConfigFormAction::StorageDir) }
|
||||
{ edit_field_text_option!(form_state, translate.t(LABEL_DEFAULT_USER_AGENT), default_user_agent, MainConfigFormAction::DefaultUserAgent) }
|
||||
{ edit_field_text_option!(form_state, translate.t(LABEL_MAPPING_PATH), mapping_path, MainConfigFormAction::MappingPath) }
|
||||
{ edit_field_text_option!(form_state, translate.t(LABEL_TEMPLATE_PATH), template_path, MainConfigFormAction::TemplatePath) }
|
||||
|
||||
@@ -70,7 +70,7 @@ generate_form_reducer!(
|
||||
fields {
|
||||
Enabled => enabled: bool,
|
||||
Size => size: Option<String>,
|
||||
Dir => dir: Option<String>,
|
||||
Dir => directory: Option<String>,
|
||||
}
|
||||
);
|
||||
|
||||
@@ -389,7 +389,7 @@ pub fn ReverseProxyConfigView() -> Html {
|
||||
<h1>{translate.t(LABEL_CACHE)}</h1>
|
||||
{ config_field_bool!(cache_state.form, translate.t(LABEL_ENABLED), enabled) }
|
||||
{ config_field_optional!(cache_state.form, translate.t(LABEL_SIZE), size) }
|
||||
{ config_field_optional!(cache_state.form, translate.t(LABEL_DIRECTORY), dir) }
|
||||
{ config_field_optional!(cache_state.form, translate.t(LABEL_DIRECTORY), directory) }
|
||||
</Card>
|
||||
}
|
||||
};
|
||||
@@ -543,7 +543,7 @@ pub fn ReverseProxyConfigView() -> Html {
|
||||
<h1>{translate.t(LABEL_CACHE)}</h1>
|
||||
{ edit_field_bool!(cache_state, translate.t(LABEL_ENABLED), enabled, CacheConfigFormAction::Enabled) }
|
||||
{ edit_field_text_option!(cache_state, translate.t(LABEL_SIZE), size, CacheConfigFormAction::Size) }
|
||||
{ edit_field_text_option!(cache_state, translate.t(LABEL_DIRECTORY), dir, CacheConfigFormAction::Dir) }
|
||||
{ edit_field_text_option!(cache_state, translate.t(LABEL_DIRECTORY), directory, CacheConfigFormAction::Dir) }
|
||||
</Card>
|
||||
}
|
||||
};
|
||||
|
||||
@@ -188,6 +188,7 @@ pub fn SchedulesConfigView() -> Html {
|
||||
<td>{ match entry.task_type {
|
||||
ScheduleTaskType::PlaylistUpdate => translate.t("LABEL.PLAYLIST_UPDATE"),
|
||||
ScheduleTaskType::LibraryScan => translate.t("LABEL.LIBRARY"),
|
||||
ScheduleTaskType::GeoIpUpdate => translate.t("LABEL.GEOIP"),
|
||||
}}</td>
|
||||
<td>
|
||||
<div class="tp__config-view__tags">
|
||||
@@ -246,6 +247,11 @@ pub fn SchedulesConfigView() -> Html {
|
||||
label: html! { translate.t("LABEL.LIBRARY") },
|
||||
selected: *selected_type == ScheduleTaskType::LibraryScan,
|
||||
},
|
||||
DropDownOption {
|
||||
id: "GeoIpUpdate".to_string(),
|
||||
label: html! { translate.t("LABEL.GEOIP") },
|
||||
selected: *selected_type == ScheduleTaskType::GeoIpUpdate,
|
||||
},
|
||||
]);
|
||||
|
||||
html! {
|
||||
@@ -260,6 +266,8 @@ pub fn SchedulesConfigView() -> Html {
|
||||
set_selected_type.set(ScheduleTaskType::PlaylistUpdate);
|
||||
} else if option == "LibraryScan" {
|
||||
set_selected_type.set(ScheduleTaskType::LibraryScan);
|
||||
} else if option == "GeoIpUpdate" {
|
||||
set_selected_type.set(ScheduleTaskType::GeoIpUpdate);
|
||||
}
|
||||
})}
|
||||
options={types}
|
||||
|
||||
@@ -179,6 +179,14 @@ generate_form_reducer!(
|
||||
}
|
||||
);
|
||||
|
||||
fn apply_parsed_input_type(staged_input_state: &UseReducerHandle<StagedInputDtoFormState>, selected: Option<&str>) {
|
||||
let input_type =
|
||||
selected.and_then(|value| value.parse::<InputType>().ok()).unwrap_or(staged_input_state.form.input_type);
|
||||
if staged_input_state.form.input_type != input_type {
|
||||
staged_input_state.dispatch(StagedInputFormAction::InputType(input_type));
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Properties, PartialEq, Clone)]
|
||||
pub struct ConfigInputViewProps {
|
||||
#[prop_or_default]
|
||||
@@ -631,16 +639,18 @@ pub fn ConfigInputView(props: &ConfigInputViewProps) -> Html {
|
||||
multi_select={false}
|
||||
on_select={Callback::from(move |(_, selections):(String, DropDownSelection)| {
|
||||
match selections {
|
||||
DropDownSelection::Empty => {
|
||||
staged_input_state_2.dispatch(StagedInputFormAction::InputType(InputType::Xtream));
|
||||
}
|
||||
DropDownSelection::Empty => {}
|
||||
DropDownSelection::Single(option) => {
|
||||
staged_input_state_2.dispatch(StagedInputFormAction::InputType(option.parse::<InputType>().unwrap_or(InputType::Xtream)));
|
||||
apply_parsed_input_type(
|
||||
&staged_input_state_2,
|
||||
Some(option.as_str()),
|
||||
);
|
||||
}
|
||||
DropDownSelection::Multi(options) => {
|
||||
if let Some(first) = options.first() {
|
||||
staged_input_state_2.dispatch(StagedInputFormAction::InputType(first.parse::<InputType>().unwrap_or(InputType::Xtream)));
|
||||
}
|
||||
apply_parsed_input_type(
|
||||
&staged_input_state_2,
|
||||
options.first().map(String::as_str),
|
||||
);
|
||||
}
|
||||
}
|
||||
})}
|
||||
@@ -896,7 +906,7 @@ pub fn ConfigInputView(props: &ConfigInputViewProps) -> Html {
|
||||
})
|
||||
}
|
||||
</div>
|
||||
<TextButton
|
||||
<TextButton
|
||||
class="primary"
|
||||
name="add_epg_source"
|
||||
icon="Add"
|
||||
|
||||
@@ -141,7 +141,6 @@ fn has_meaningful_dns_config(dns: &ProviderDnsDto) -> bool {
|
||||
|| dns.schemes.as_ref().is_some_and(|schemes| !schemes.is_empty())
|
||||
|| dns.keep_vhost
|
||||
|| dns.overrides.as_ref().is_some_and(|overrides| !overrides.is_empty())
|
||||
|| dns.resolved.as_ref().is_some_and(|resolved| !resolved.is_empty())
|
||||
|| dns.on_resolve_error != OnResolveErrorPolicy::default()
|
||||
|| dns.on_connect_error != OnConnectErrorPolicy::default()
|
||||
}
|
||||
|
||||
@@ -802,33 +802,23 @@ impl<'a> MapperContext<'a> {
|
||||
BuiltInFunction::ToNumber
|
||||
| BuiltInFunction::Template
|
||||
| BuiltInFunction::First
|
||||
| BuiltInFunction::AddFavourite => {
|
||||
if args.len() > 1 {
|
||||
return info_err_res!(
|
||||
"Function accepts only one argument {:?}, {} given",
|
||||
name,
|
||||
args.len()
|
||||
);
|
||||
}
|
||||
| BuiltInFunction::AddFavourite
|
||||
if args.len() > 1 =>
|
||||
{
|
||||
return info_err_res!("Function accepts only one argument {:?}, {} given", name, args.len());
|
||||
}
|
||||
BuiltInFunction::Split => {
|
||||
if args.len() != 2 {
|
||||
return info_err_res!("Function accepts two arguments {:?}, {} given", name, args.len());
|
||||
}
|
||||
BuiltInFunction::Split if args.len() != 2 => {
|
||||
return info_err_res!("Function accepts two arguments {:?}, {} given", name, args.len());
|
||||
}
|
||||
BuiltInFunction::Replace => {
|
||||
if args.len() != 3 {
|
||||
return info_err_res!("Function accepts three arguments {:?}, {} given", name, args.len());
|
||||
}
|
||||
BuiltInFunction::Replace if args.len() != 3 => {
|
||||
return info_err_res!("Function accepts three arguments {:?}, {} given", name, args.len());
|
||||
}
|
||||
BuiltInFunction::Pad => {
|
||||
if !(args.len() == 3 || args.len() == 4) {
|
||||
return info_err_res!(
|
||||
"Function accepts three or four arguments {:?}, {} given",
|
||||
name,
|
||||
args.len()
|
||||
);
|
||||
}
|
||||
BuiltInFunction::Pad if !(args.len() == 3 || args.len() == 4) => {
|
||||
return info_err_res!(
|
||||
"Function accepts three or four arguments {:?}, {} given",
|
||||
name,
|
||||
args.len()
|
||||
);
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
|
||||
@@ -1,8 +1,52 @@
|
||||
use crate::utils::{get_default_web_root, is_blank_or_default_web_root};
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, Default, PartialEq)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct ConfigApiDto {
|
||||
pub host: String,
|
||||
pub port: u16,
|
||||
#[serde(default, skip_serializing_if = "String::is_empty")]
|
||||
#[serde(default, skip_serializing_if = "is_blank_or_default_web_root")]
|
||||
pub web_root: String,
|
||||
}
|
||||
|
||||
impl ConfigApiDto {
|
||||
pub fn default_web_root() -> String { get_default_web_root() }
|
||||
|
||||
pub fn prepare(&mut self) {
|
||||
if self.web_root.trim().is_empty() {
|
||||
self.web_root = Self::default_web_root();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::ConfigApiDto;
|
||||
|
||||
#[test]
|
||||
fn serialization_skips_default_web_root() {
|
||||
let dto = ConfigApiDto { host: "0.0.0.0".to_string(), port: 8901, web_root: "./web".to_string() };
|
||||
|
||||
let serialized = serde_json::to_string(&dto).expect("api dto serialization should succeed");
|
||||
assert!(!serialized.contains("\"web_root\""), "expected default web_root to be skipped, got: {serialized}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serialization_keeps_non_default_web_root() {
|
||||
let dto = ConfigApiDto { host: "0.0.0.0".to_string(), port: 8901, web_root: "/srv/tuliprox/web".to_string() };
|
||||
|
||||
let serialized = serde_json::to_string(&dto).expect("api dto serialization should succeed");
|
||||
assert!(
|
||||
serialized.contains("\"web_root\""),
|
||||
"expected non-default web_root to be persisted, got: {serialized}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prepare_sets_default_web_root_when_empty() {
|
||||
let mut dto = ConfigApiDto { host: "0.0.0.0".to_string(), port: 8901, web_root: String::new() };
|
||||
let expected_web_root = ConfigApiDto::default_web_root();
|
||||
dto.prepare();
|
||||
assert_eq!(dto.web_root, expected_web_root);
|
||||
}
|
||||
}
|
||||
|
||||
+200
-69
@@ -6,55 +6,18 @@ use crate::{
|
||||
WebUiConfigDto,
|
||||
},
|
||||
utils::{
|
||||
default_connect_timeout_secs, default_supported_video_extensions, is_blank_optional_string,
|
||||
is_default_connect_timeout_secs, is_false, CONFIG_PATH, MAPPING_FILE, TEMPLATE_FILE,
|
||||
default_connect_timeout_secs, default_custom_stream_response_path, default_default_user_agent,
|
||||
default_main_backup_dir, default_main_mapping_path, default_main_storage_dir, default_main_template_path,
|
||||
default_main_user_config_dir, default_supported_video_extensions, is_blank_optional_string,
|
||||
is_blank_or_default_backup_dir, is_blank_or_default_custom_stream_response_path,
|
||||
is_blank_or_default_mapping_path, is_blank_or_default_storage_dir, is_blank_or_default_template_path,
|
||||
is_blank_or_default_user_config_dir, is_default_connect_timeout_secs, is_false,
|
||||
is_none_or_empty_metadata_update, is_none_or_empty_video, normalize_optional_config_file_path,
|
||||
normalize_optional_dir, DEFAULT_BACKUP_DIR, DEFAULT_CUSTOM_STREAM_RESPONSE_PATH, DEFAULT_STORAGE_DIR,
|
||||
DEFAULT_USER_CONFIG_DIR, MAPPING_FILE, TEMPLATE_FILE,
|
||||
},
|
||||
};
|
||||
|
||||
pub const DEFAULT_USER_AGENT: &str = "VLC/3.0.16 LibVLC/3.0.16";
|
||||
|
||||
fn default_default_user_agent() -> Option<String> { Some(DEFAULT_USER_AGENT.to_string()) }
|
||||
fn default_main_mapping_path() -> Option<String> { Some(format!("./{CONFIG_PATH}/{MAPPING_FILE}")) }
|
||||
fn default_main_template_path() -> Option<String> { Some(format!("./{CONFIG_PATH}/{TEMPLATE_FILE}")) }
|
||||
|
||||
fn is_default_config_file_path(value: &str, file_name: &str) -> bool {
|
||||
let normalized = value.trim().replace('\\', "/");
|
||||
let normalized = normalized.trim_start_matches("./");
|
||||
normalized == file_name
|
||||
|| normalized.rsplit_once('/').is_some_and(|(dir, file)| dir == CONFIG_PATH && file == file_name)
|
||||
}
|
||||
|
||||
fn is_blank_or_default_mapping_path(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || is_default_config_file_path(value, MAPPING_FILE))
|
||||
}
|
||||
|
||||
fn is_blank_or_default_template_path(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || is_default_config_file_path(value, TEMPLATE_FILE))
|
||||
}
|
||||
|
||||
fn effective_main_mapping_path(path: Option<String>) -> Option<String> {
|
||||
if is_blank_or_default_mapping_path(&path) {
|
||||
default_main_mapping_path()
|
||||
} else {
|
||||
path
|
||||
}
|
||||
}
|
||||
|
||||
fn effective_main_template_path(path: Option<String>) -> Option<String> {
|
||||
if is_blank_or_default_template_path(&path) {
|
||||
default_main_template_path()
|
||||
} else {
|
||||
path
|
||||
}
|
||||
}
|
||||
|
||||
fn is_none_or_empty_video(video: &Option<VideoConfigDto>) -> bool {
|
||||
video.as_ref().is_none_or(VideoConfigDto::is_empty)
|
||||
}
|
||||
fn is_none_or_empty_metadata_update(metadata_update: &Option<MetadataUpdateConfigDto>) -> bool {
|
||||
metadata_update.as_ref().is_none_or(MetadataUpdateConfigDto::is_empty)
|
||||
}
|
||||
|
||||
#[allow(clippy::struct_excessive_bools)]
|
||||
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, PartialEq)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
@@ -62,18 +25,22 @@ pub struct ConfigDto {
|
||||
#[serde(default, skip_serializing_if = "is_false")]
|
||||
pub process_parallel: bool,
|
||||
pub api: ConfigApiDto,
|
||||
pub working_dir: String,
|
||||
#[serde(default, alias = "working_dir", skip_serializing_if = "is_blank_or_default_storage_dir")]
|
||||
pub storage_dir: Option<String>,
|
||||
#[serde(default = "default_default_user_agent", skip_serializing_if = "is_blank_optional_string")]
|
||||
pub default_user_agent: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
#[serde(default, skip_serializing_if = "is_blank_or_default_backup_dir")]
|
||||
pub backup_dir: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
#[serde(default, skip_serializing_if = "is_blank_or_default_user_config_dir")]
|
||||
pub user_config_dir: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "is_blank_or_default_mapping_path")]
|
||||
pub mapping_path: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "is_blank_or_default_template_path")]
|
||||
pub template_path: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
#[serde(
|
||||
default = "default_custom_stream_response_path",
|
||||
skip_serializing_if = "is_blank_or_default_custom_stream_response_path"
|
||||
)]
|
||||
pub custom_stream_response_path: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "is_none_or_empty_video")]
|
||||
pub video: Option<VideoConfigDto>,
|
||||
@@ -118,7 +85,7 @@ impl Default for ConfigDto {
|
||||
Self {
|
||||
process_parallel: false,
|
||||
api: ConfigApiDto::default(),
|
||||
working_dir: String::new(),
|
||||
storage_dir: None,
|
||||
default_user_agent: default_default_user_agent(),
|
||||
backup_dir: None,
|
||||
user_config_dir: None,
|
||||
@@ -153,12 +120,13 @@ impl Default for ConfigDto {
|
||||
pub struct MainConfigDto {
|
||||
#[serde(default, skip_serializing_if = "is_false")]
|
||||
pub process_parallel: bool,
|
||||
pub working_dir: String,
|
||||
#[serde(default = "default_main_storage_dir", skip_serializing_if = "is_blank_or_default_storage_dir")]
|
||||
pub storage_dir: Option<String>,
|
||||
#[serde(default = "default_default_user_agent", skip_serializing_if = "is_blank_optional_string")]
|
||||
pub default_user_agent: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
#[serde(default = "default_main_backup_dir", skip_serializing_if = "is_blank_or_default_backup_dir")]
|
||||
pub backup_dir: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
#[serde(default = "default_main_user_config_dir", skip_serializing_if = "is_blank_or_default_user_config_dir")]
|
||||
pub user_config_dir: Option<String>,
|
||||
#[serde(default = "default_main_mapping_path", skip_serializing_if = "is_blank_or_default_mapping_path")]
|
||||
pub mapping_path: Option<String>,
|
||||
@@ -187,10 +155,10 @@ impl Default for MainConfigDto {
|
||||
MainConfigDto {
|
||||
process_parallel: false,
|
||||
disk_based_processing: false,
|
||||
working_dir: String::new(),
|
||||
storage_dir: default_main_storage_dir(),
|
||||
default_user_agent: default_default_user_agent(),
|
||||
backup_dir: None,
|
||||
user_config_dir: None,
|
||||
backup_dir: default_main_backup_dir(),
|
||||
user_config_dir: default_main_user_config_dir(),
|
||||
mapping_path: default_main_mapping_path(),
|
||||
template_path: default_main_template_path(),
|
||||
custom_stream_response_path: None,
|
||||
@@ -209,12 +177,12 @@ impl From<&ConfigDto> for MainConfigDto {
|
||||
Self {
|
||||
process_parallel: config.process_parallel,
|
||||
disk_based_processing: config.disk_based_processing,
|
||||
working_dir: config.working_dir.clone(),
|
||||
storage_dir: config.storage_dir.clone(),
|
||||
default_user_agent: config.default_user_agent.clone(),
|
||||
backup_dir: config.backup_dir.clone(),
|
||||
user_config_dir: config.user_config_dir.clone(),
|
||||
mapping_path: effective_main_mapping_path(config.mapping_path.clone()),
|
||||
template_path: effective_main_template_path(config.template_path.clone()),
|
||||
mapping_path: config.mapping_path.clone(),
|
||||
template_path: config.template_path.clone(),
|
||||
custom_stream_response_path: config.custom_stream_response_path.clone(),
|
||||
user_access_control: config.user_access_control,
|
||||
connect_timeout_secs: config.connect_timeout_secs,
|
||||
@@ -249,9 +217,26 @@ pub struct HdHomeRunDeviceOverview {
|
||||
|
||||
impl ConfigDto {
|
||||
pub fn prepare(&mut self, include_computed: bool) -> Result<(), TuliproxError> {
|
||||
self.api.prepare();
|
||||
|
||||
if is_blank_optional_string(&self.default_user_agent) {
|
||||
self.default_user_agent = default_default_user_agent();
|
||||
}
|
||||
if is_blank_or_default_storage_dir(&self.storage_dir) {
|
||||
self.storage_dir = default_main_storage_dir();
|
||||
}
|
||||
if is_blank_or_default_backup_dir(&self.backup_dir) {
|
||||
self.backup_dir = default_main_backup_dir();
|
||||
}
|
||||
if is_blank_or_default_user_config_dir(&self.user_config_dir) {
|
||||
self.user_config_dir = default_main_user_config_dir();
|
||||
}
|
||||
if is_blank_or_default_mapping_path(&self.mapping_path) {
|
||||
self.mapping_path = default_main_mapping_path();
|
||||
}
|
||||
if is_blank_or_default_template_path(&self.template_path) {
|
||||
self.template_path = default_main_template_path();
|
||||
}
|
||||
|
||||
if let Some(mins) = self.sleep_timer_mins {
|
||||
if mins == 0 {
|
||||
@@ -268,7 +253,7 @@ impl ConfigDto {
|
||||
self.prepare_metadata_update_config()?;
|
||||
|
||||
if let Some(reverse_proxy) = self.reverse_proxy.as_mut() {
|
||||
reverse_proxy.prepare(&self.working_dir)?;
|
||||
reverse_proxy.prepare(self.storage_dir.as_deref().unwrap_or_default())?;
|
||||
}
|
||||
if let Some(proxy) = &mut self.proxy {
|
||||
proxy.prepare()?;
|
||||
@@ -280,6 +265,9 @@ impl ConfigDto {
|
||||
if let Some(messaging) = &mut self.messaging {
|
||||
messaging.prepare(include_computed)?;
|
||||
}
|
||||
if let Some(library) = &mut self.library {
|
||||
library.playlist.prepare();
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -361,13 +349,14 @@ impl ConfigDto {
|
||||
pub fn update_from_main_config(&mut self, main_config: &MainConfigDto) {
|
||||
self.process_parallel = main_config.process_parallel;
|
||||
self.disk_based_processing = main_config.disk_based_processing;
|
||||
self.working_dir = main_config.working_dir.clone();
|
||||
self.storage_dir = normalize_optional_dir(&main_config.storage_dir, DEFAULT_STORAGE_DIR);
|
||||
self.default_user_agent = main_config.default_user_agent.clone();
|
||||
self.backup_dir = main_config.backup_dir.clone();
|
||||
self.user_config_dir = main_config.user_config_dir.clone();
|
||||
self.mapping_path = main_config.mapping_path.clone();
|
||||
self.template_path = main_config.template_path.clone();
|
||||
self.custom_stream_response_path = main_config.custom_stream_response_path.clone();
|
||||
self.backup_dir = normalize_optional_dir(&main_config.backup_dir, DEFAULT_BACKUP_DIR);
|
||||
self.user_config_dir = normalize_optional_dir(&main_config.user_config_dir, DEFAULT_USER_CONFIG_DIR);
|
||||
self.mapping_path = normalize_optional_config_file_path(&main_config.mapping_path, MAPPING_FILE);
|
||||
self.template_path = normalize_optional_config_file_path(&main_config.template_path, TEMPLATE_FILE);
|
||||
self.custom_stream_response_path =
|
||||
normalize_optional_dir(&main_config.custom_stream_response_path, DEFAULT_CUSTOM_STREAM_RESPONSE_PATH);
|
||||
self.user_access_control = main_config.user_access_control;
|
||||
self.connect_timeout_secs = main_config.connect_timeout_secs;
|
||||
self.sleep_timer_mins = main_config.sleep_timer_mins;
|
||||
@@ -386,7 +375,7 @@ impl ConfigDto {
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::utils::default_supported_video_extensions;
|
||||
use crate::utils::{default_supported_video_extensions, CONFIG_PATH};
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
@@ -425,6 +414,104 @@ mod tests {
|
||||
assert!(serialized.contains("\"video\""), "expected video field, got: {serialized}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serializing_skips_default_storage_backup_and_user_config_dirs() {
|
||||
let cfg = ConfigDto {
|
||||
storage_dir: Some(DEFAULT_STORAGE_DIR.to_string()),
|
||||
backup_dir: Some(DEFAULT_BACKUP_DIR.to_string()),
|
||||
user_config_dir: Some(DEFAULT_USER_CONFIG_DIR.to_string()),
|
||||
..ConfigDto::default()
|
||||
};
|
||||
|
||||
let serialized = serde_json::to_string(&cfg).expect("config serialization should succeed");
|
||||
assert!(
|
||||
!serialized.contains("\"storage_dir\""),
|
||||
"expected no storage_dir field for default value, got: {serialized}"
|
||||
);
|
||||
assert!(
|
||||
!serialized.contains("\"backup_dir\""),
|
||||
"expected no backup_dir field for default value, got: {serialized}"
|
||||
);
|
||||
assert!(
|
||||
!serialized.contains("\"user_config_dir\""),
|
||||
"expected no user_config_dir field for default value, got: {serialized}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serializing_keeps_non_default_storage_and_backup_dirs() {
|
||||
let cfg = ConfigDto {
|
||||
storage_dir: Some("custom-storage".to_string()),
|
||||
backup_dir: Some("custom-backup".to_string()),
|
||||
user_config_dir: Some("custom-user-config".to_string()),
|
||||
..ConfigDto::default()
|
||||
};
|
||||
|
||||
let serialized = serde_json::to_string(&cfg).expect("config serialization should succeed");
|
||||
assert!(
|
||||
serialized.contains("\"storage_dir\""),
|
||||
"expected storage_dir field for non-default value, got: {serialized}"
|
||||
);
|
||||
assert!(
|
||||
serialized.contains("\"backup_dir\""),
|
||||
"expected backup_dir field for non-default value, got: {serialized}"
|
||||
);
|
||||
assert!(
|
||||
serialized.contains("\"user_config_dir\""),
|
||||
"expected user_config_dir field for non-default value, got: {serialized}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn main_config_from_applies_default_storage_backup_and_user_config_dirs() {
|
||||
let mut cfg = ConfigDto::default();
|
||||
cfg.prepare(false).expect("prepare should succeed");
|
||||
let main = MainConfigDto::from(&cfg);
|
||||
assert_eq!(main.storage_dir.as_deref(), Some(DEFAULT_STORAGE_DIR));
|
||||
assert_eq!(main.backup_dir.as_deref(), Some(DEFAULT_BACKUP_DIR));
|
||||
assert_eq!(main.user_config_dir.as_deref(), Some(DEFAULT_USER_CONFIG_DIR));
|
||||
assert_eq!(main.mapping_path.as_deref(), Some(format!("./{CONFIG_PATH}/{MAPPING_FILE}").as_str()));
|
||||
assert_eq!(main.template_path.as_deref(), Some(format!("./{CONFIG_PATH}/{TEMPLATE_FILE}").as_str()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn update_from_main_config_omits_default_optional_paths() {
|
||||
let mut cfg = ConfigDto::default();
|
||||
let main = MainConfigDto {
|
||||
storage_dir: Some(DEFAULT_STORAGE_DIR.to_string()),
|
||||
backup_dir: Some(DEFAULT_BACKUP_DIR.to_string()),
|
||||
user_config_dir: Some(DEFAULT_USER_CONFIG_DIR.to_string()),
|
||||
mapping_path: Some(format!("./{CONFIG_PATH}/{MAPPING_FILE}")),
|
||||
template_path: Some(format!("./{CONFIG_PATH}/{TEMPLATE_FILE}")),
|
||||
..MainConfigDto::default()
|
||||
};
|
||||
|
||||
cfg.update_from_main_config(&main);
|
||||
assert!(cfg.storage_dir.is_none());
|
||||
assert!(cfg.backup_dir.is_none());
|
||||
assert!(cfg.user_config_dir.is_none());
|
||||
assert!(cfg.mapping_path.is_none());
|
||||
assert!(cfg.template_path.is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prepare_sets_default_optional_paths() {
|
||||
let mut cfg = ConfigDto {
|
||||
storage_dir: None,
|
||||
backup_dir: None,
|
||||
user_config_dir: None,
|
||||
mapping_path: None,
|
||||
template_path: None,
|
||||
..ConfigDto::default()
|
||||
};
|
||||
cfg.prepare(false).expect("prepare should succeed");
|
||||
assert_eq!(cfg.storage_dir.as_deref(), Some(DEFAULT_STORAGE_DIR));
|
||||
assert_eq!(cfg.backup_dir.as_deref(), Some(DEFAULT_BACKUP_DIR));
|
||||
assert_eq!(cfg.user_config_dir.as_deref(), Some(DEFAULT_USER_CONFIG_DIR));
|
||||
assert_eq!(cfg.mapping_path.as_deref(), Some(format!("./{CONFIG_PATH}/{MAPPING_FILE}").as_str()));
|
||||
assert_eq!(cfg.template_path.as_deref(), Some(format!("./{CONFIG_PATH}/{TEMPLATE_FILE}").as_str()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deserializing_rejects_legacy_video_ffprobe_fields() {
|
||||
let raw = json!({
|
||||
@@ -433,7 +520,7 @@ mod tests {
|
||||
"port": 8901,
|
||||
"web_root": "./web"
|
||||
},
|
||||
"working_dir": ".",
|
||||
"storage_dir": ".",
|
||||
"video": {
|
||||
"extensions": ["mp4"],
|
||||
"ffprobe_enabled": true
|
||||
@@ -445,4 +532,48 @@ mod tests {
|
||||
let err = result.unwrap_err().to_string();
|
||||
assert!(err.contains("ffprobe_enabled"), "unexpected error text: {err}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deserializing_rejects_legacy_data_dir_alias() {
|
||||
let raw = json!({
|
||||
"api": {
|
||||
"host": "127.0.0.1",
|
||||
"port": 8901,
|
||||
"web_root": "./web"
|
||||
},
|
||||
"data_dir": "."
|
||||
});
|
||||
|
||||
let result: Result<ConfigDto, _> = serde_json::from_value(raw);
|
||||
assert!(result.is_err(), "data_dir should not deserialize");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deserializing_accepts_legacy_working_dir_alias() {
|
||||
let raw = json!({
|
||||
"api": {
|
||||
"host": "127.0.0.1",
|
||||
"port": 8901,
|
||||
"web_root": "./web"
|
||||
},
|
||||
"working_dir": "."
|
||||
});
|
||||
|
||||
let cfg: ConfigDto = serde_json::from_value(raw).expect("working_dir should deserialize as legacy alias");
|
||||
assert_eq!(cfg.storage_dir.as_deref(), Some("."));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deserializing_accepts_missing_storage_dir() {
|
||||
let raw = json!({
|
||||
"api": {
|
||||
"host": "127.0.0.1",
|
||||
"port": 8901,
|
||||
"web_root": "./web"
|
||||
}
|
||||
});
|
||||
|
||||
let cfg: ConfigDto = serde_json::from_value(raw).expect("missing storage_dir should deserialize");
|
||||
assert!(cfg.storage_dir.is_none());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
use crate::{
|
||||
error::TuliproxError,
|
||||
info_err_res,
|
||||
utils::{is_blank_optional_str, is_blank_optional_string, parse_size_base_2},
|
||||
utils::{
|
||||
is_blank_optional_str, is_blank_optional_string, is_blank_or_default_cache_dir, parse_size_base_2,
|
||||
DEFAULT_CACHE_DIR,
|
||||
},
|
||||
};
|
||||
use path_clean::PathClean;
|
||||
use std::path::PathBuf;
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, Default, PartialEq)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
@@ -13,27 +14,21 @@ pub struct CacheConfigDto {
|
||||
pub enabled: bool,
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
pub size: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
pub dir: Option<String>,
|
||||
#[serde(default, alias = "dir", skip_serializing_if = "is_blank_or_default_cache_dir")]
|
||||
pub directory: Option<String>,
|
||||
}
|
||||
|
||||
impl CacheConfigDto {
|
||||
pub fn is_empty(&self) -> bool {
|
||||
!self.enabled && is_blank_optional_str(self.size.as_deref()) && is_blank_optional_str(self.dir.as_deref())
|
||||
!self.enabled && is_blank_optional_str(self.size.as_deref()) && is_blank_optional_str(self.directory.as_deref())
|
||||
}
|
||||
|
||||
pub(crate) fn prepare(&mut self, working_dir: &str) -> Result<(), TuliproxError> {
|
||||
pub(crate) fn prepare(&mut self, _storage_dir: &str) -> Result<(), TuliproxError> {
|
||||
if self.enabled {
|
||||
let work_path = PathBuf::from(working_dir);
|
||||
match self.dir.as_ref() {
|
||||
None => self.dir = Some(work_path.join("cache").to_string_lossy().to_string()),
|
||||
Some(work_dir) => {
|
||||
let mut cache_dir = work_dir.to_string();
|
||||
if PathBuf::from(&cache_dir).is_relative() {
|
||||
cache_dir = work_path.join(&cache_dir).clean().to_string_lossy().to_string();
|
||||
}
|
||||
self.dir = Some(cache_dir.to_string());
|
||||
}
|
||||
if is_blank_or_default_cache_dir(&self.directory) {
|
||||
self.directory = Some(DEFAULT_CACHE_DIR.to_string());
|
||||
} else if let Some(dir) = self.directory.as_ref() {
|
||||
self.directory = Some(dir.trim().to_string());
|
||||
}
|
||||
|
||||
if let Some(val) = self.size.as_ref() {
|
||||
@@ -50,3 +45,36 @@ impl CacheConfigDto {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn prepare_sets_default_cache_dir_when_enabled_and_missing() {
|
||||
let mut cache = CacheConfigDto { enabled: true, directory: None, size: None };
|
||||
cache.prepare("storage").expect("prepare should succeed");
|
||||
assert_eq!(cache.directory.as_deref(), Some(DEFAULT_CACHE_DIR));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prepare_keeps_custom_cache_dir_when_enabled() {
|
||||
let mut cache = CacheConfigDto { enabled: true, directory: Some("custom-cache".to_string()), size: None };
|
||||
cache.prepare("storage").expect("prepare should succeed");
|
||||
assert_eq!(cache.directory.as_deref(), Some("custom-cache"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serializing_skips_default_cache_dir() {
|
||||
let cache = CacheConfigDto { enabled: true, directory: Some(DEFAULT_CACHE_DIR.to_string()), size: None };
|
||||
let serialized = serde_json::to_string(&cache).expect("cache serialization should succeed");
|
||||
assert!(!serialized.contains("\"directory\""), "expected no dir field for default value, got: {serialized}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serializing_keeps_non_default_cache_dir() {
|
||||
let cache = CacheConfigDto { enabled: true, directory: Some("custom-cache".to_string()), size: None };
|
||||
let serialized = serde_json::to_string(&cache).expect("cache serialization should succeed");
|
||||
assert!(serialized.contains("\"directory\""), "expected dir field for custom value, got: {serialized}");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -10,11 +10,10 @@ use crate::{
|
||||
deserialize_timestamp, get_credentials_from_url_str, get_trimmed_string, is_blank_optional_string,
|
||||
is_default_probe_delay_secs, is_default_probe_live_interval, is_default_resolve_delay_secs, is_false, is_true,
|
||||
is_zero_i16, is_zero_u16, parse_duration_seconds, parse_provider_scheme_url_parts, sanitize_sensitive_info,
|
||||
serialize_option_vec_flow_map_items, trim_last_slash, Internable, PROVIDER_SCHEME_PREFIX,
|
||||
serialize_option_vec_flow_map_items, trim_last_slash, Internable, BATCH_SCHEME_PREFIX, PROVIDER_SCHEME_PREFIX,
|
||||
},
|
||||
};
|
||||
use enum_iterator::Sequence;
|
||||
use indexmap::IndexMap;
|
||||
use log::warn;
|
||||
use std::{
|
||||
collections::{HashMap, HashSet},
|
||||
@@ -511,7 +510,7 @@ impl ConfigInputDto {
|
||||
|
||||
self.persist = get_trimmed_string(self.persist.as_deref());
|
||||
|
||||
if self.url.starts_with("file://") {
|
||||
if self.url.starts_with(BATCH_SCHEME_PREFIX) {
|
||||
match self.input_type {
|
||||
InputType::M3u => {
|
||||
self.input_type = InputType::M3uBatch;
|
||||
@@ -524,7 +523,10 @@ impl ConfigInputDto {
|
||||
} else if self.url.starts_with(PROVIDER_SCHEME_PREFIX)
|
||||
&& matches!(self.input_type, InputType::M3uBatch | InputType::XtreamBatch)
|
||||
{
|
||||
return info_err_res!("input type {} does not support provider:// URLs for batch definitions; use a local CSV path or file:// URL", self.input_type);
|
||||
return info_err_res!(
|
||||
"input type {} does not support provider:// URLs for batch definitions; use batch:// URL",
|
||||
self.input_type
|
||||
);
|
||||
}
|
||||
check_provider_scheme_url!(self.url, provider_names);
|
||||
|
||||
@@ -654,7 +656,10 @@ impl ConfigInputDto {
|
||||
if self.url.starts_with(PROVIDER_SCHEME_PREFIX)
|
||||
&& matches!(self.input_type, InputType::M3uBatch | InputType::XtreamBatch)
|
||||
{
|
||||
return info_err_res!("input type {} does not support provider:// URLs for batch definitions; use a local CSV path or file:// URL", self.input_type);
|
||||
return info_err_res!(
|
||||
"input type {} does not support provider:// URLs for batch definitions; use batch:// URL",
|
||||
self.input_type
|
||||
);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -791,8 +796,6 @@ pub struct ProviderDnsDto {
|
||||
pub keep_vhost: bool,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub overrides: Option<HashMap<String, Vec<IpAddr>>>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub resolved: Option<IndexMap<String, Vec<IpAddr>>>,
|
||||
#[serde(default, skip_serializing_if = "is_default_on_resolve_error")]
|
||||
pub on_resolve_error: OnResolveErrorPolicy,
|
||||
#[serde(default, skip_serializing_if = "is_default_on_connect_error")]
|
||||
@@ -809,7 +812,6 @@ impl Default for ProviderDnsDto {
|
||||
schemes: None,
|
||||
keep_vhost: false,
|
||||
overrides: None,
|
||||
resolved: None,
|
||||
on_resolve_error: OnResolveErrorPolicy::default(),
|
||||
on_connect_error: OnConnectErrorPolicy::default(),
|
||||
}
|
||||
@@ -1010,29 +1012,12 @@ mod tests {
|
||||
assert_eq!(overrides["example.com"].len(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_provider_dns_resolved_deserializes() {
|
||||
let json = r#"{
|
||||
"name":"p1",
|
||||
"urls":["http://example.com"],
|
||||
"dns":{
|
||||
"enabled":true,
|
||||
"resolved":{"example.com":["203.0.113.10"]}
|
||||
}
|
||||
}"#;
|
||||
|
||||
let dto: ConfigProviderDto = serde_json::from_str(json).expect("provider json should parse");
|
||||
let dns = dto.dns.expect("dns should be present");
|
||||
let resolved = dns.resolved.expect("resolved must be deserialized");
|
||||
assert_eq!(resolved.get("example.com"), Some(&vec!["203.0.113.10".parse::<IpAddr>().expect("valid ip")]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prepare_switches_xtream_to_xtream_batch_when_alias_exists() {
|
||||
let mut dto = ConfigInputDto {
|
||||
name: "input_alias".intern(),
|
||||
input_type: InputType::Xtream,
|
||||
url: "file:///tmp/input_alias.csv".to_string(),
|
||||
url: "batch:///tmp/input_alias.csv".to_string(),
|
||||
aliases: Some(vec![ConfigInputAliasDto {
|
||||
id: 1,
|
||||
name: "alias_1".intern(),
|
||||
@@ -1046,7 +1031,7 @@ mod tests {
|
||||
};
|
||||
|
||||
dto.prepare_type().expect("prepare type should succeed");
|
||||
dto.prepare(0, true, &HashSet::new()).expect("prepare should succeed and infer batch type");
|
||||
dto.prepare(0, true, &HashSet::new()).expect("prepare should succeed and infer batch type from batch:// URL");
|
||||
assert_eq!(dto.input_type, InputType::XtreamBatch);
|
||||
}
|
||||
|
||||
@@ -1069,6 +1054,6 @@ mod tests {
|
||||
};
|
||||
|
||||
let err = dto.prepare(0, true, &HashSet::new()).expect_err("prepare must reject provider:// for batch input");
|
||||
assert!(err.to_string().contains("does not support provider:// URLs"));
|
||||
assert!(err.to_string().contains("does not support provider:// URLs"), "Error: {err}");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -106,7 +106,7 @@ pub enum LibraryMetadataFormat {
|
||||
Nfo,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Default)]
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct LibraryPlaylistConfigDto {
|
||||
#[serde(default = "default_movie_category")]
|
||||
@@ -119,7 +119,7 @@ impl LibraryPlaylistConfigDto {
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.movie_category == default_movie_category() && self.series_category == default_series_category()
|
||||
}
|
||||
pub fn clean(&mut self) {
|
||||
pub fn prepare(&mut self) {
|
||||
if self.movie_category.trim().is_empty() {
|
||||
self.movie_category = default_movie_category();
|
||||
}
|
||||
@@ -127,10 +127,19 @@ impl LibraryPlaylistConfigDto {
|
||||
self.series_category = default_series_category();
|
||||
}
|
||||
}
|
||||
pub fn clean(&mut self) { self.prepare(); }
|
||||
}
|
||||
|
||||
impl Default for LibraryPlaylistConfigDto {
|
||||
fn default() -> Self {
|
||||
Self { movie_category: default_movie_category(), series_category: default_series_category() }
|
||||
}
|
||||
}
|
||||
|
||||
impl LibraryConfigDto {
|
||||
pub fn prepare(&mut self) -> Result<(), TuliproxError> {
|
||||
self.playlist.prepare();
|
||||
|
||||
// Validate enabled state
|
||||
if self.enabled && self.scan_directories.is_empty() {
|
||||
return info_err_res!("Library enabled but no scan_directories configured");
|
||||
@@ -146,3 +155,24 @@ impl LibraryConfigDto {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{default_movie_category, default_series_category, LibraryPlaylistConfigDto};
|
||||
|
||||
#[test]
|
||||
fn playlist_default_uses_default_categories() {
|
||||
let playlist = LibraryPlaylistConfigDto::default();
|
||||
assert_eq!(playlist.movie_category, default_movie_category());
|
||||
assert_eq!(playlist.series_category, default_series_category());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn playlist_prepare_sets_default_categories_when_empty() {
|
||||
let mut playlist =
|
||||
LibraryPlaylistConfigDto { movie_category: String::new(), series_category: " ".to_string() };
|
||||
playlist.prepare();
|
||||
assert_eq!(playlist.movie_category, default_movie_category());
|
||||
assert_eq!(playlist.series_category, default_series_category());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,7 +2,11 @@ use crate::utils::{is_blank_optional_string, is_empty_optional_vec};
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
|
||||
pub struct ConfigPaths {
|
||||
#[serde(default, skip_serializing_if = "String::is_empty")]
|
||||
pub home_path: String,
|
||||
pub config_path: String,
|
||||
#[serde(default, skip_serializing_if = "String::is_empty")]
|
||||
pub storage_path: String,
|
||||
pub config_file_path: String,
|
||||
pub sources_file_path: String,
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
|
||||
@@ -89,7 +89,7 @@ impl ReverseProxyConfigDto {
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn prepare(&mut self, working_dir: &str) -> Result<(), TuliproxError> {
|
||||
pub(crate) fn prepare(&mut self, storage_dir: &str) -> Result<(), TuliproxError> {
|
||||
self.rewrite_secret = self.rewrite_secret.trim().to_string();
|
||||
if !self.resource_rewrite_disabled {
|
||||
if self.rewrite_secret.is_empty() {
|
||||
@@ -106,7 +106,7 @@ impl ReverseProxyConfigDto {
|
||||
warn!("The cache is disabled because resource rewrite is disabled");
|
||||
cache.enabled = false;
|
||||
}
|
||||
cache.prepare(working_dir)?;
|
||||
cache.prepare(storage_dir)?;
|
||||
}
|
||||
|
||||
if let Some(rate_limit) = self.rate_limit.as_mut() {
|
||||
|
||||
@@ -3,6 +3,51 @@ pub enum ScheduleTaskType {
|
||||
#[default]
|
||||
PlaylistUpdate,
|
||||
LibraryScan,
|
||||
GeoIpUpdate,
|
||||
}
|
||||
|
||||
const fn default_schedule_task_type() -> ScheduleTaskType { ScheduleTaskType::PlaylistUpdate }
|
||||
|
||||
fn parse_schedule_task_type(value: &str) -> Option<ScheduleTaskType> {
|
||||
match value.trim().to_ascii_lowercase().as_str() {
|
||||
"playlistupdate" | "playlist_update" | "playlist-update" | "update" => Some(ScheduleTaskType::PlaylistUpdate),
|
||||
"libraryscan" | "library_scan" | "library-scan" | "scan" => Some(ScheduleTaskType::LibraryScan),
|
||||
"geoipupdate" | "geo_ip_update" | "geoip_update" | "geoip-update" | "geoip" => {
|
||||
Some(ScheduleTaskType::GeoIpUpdate)
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn deserialize_schedule_task_type<'de, D>(deserializer: D) -> Result<ScheduleTaskType, D::Error>
|
||||
where
|
||||
D: serde::Deserializer<'de>,
|
||||
{
|
||||
use serde::Deserialize as _;
|
||||
|
||||
#[derive(serde::Deserialize)]
|
||||
#[serde(untagged)]
|
||||
enum CompatTaskType {
|
||||
Enum(ScheduleTaskType),
|
||||
String(String),
|
||||
}
|
||||
|
||||
let value = Option::<CompatTaskType>::deserialize(deserializer)?;
|
||||
match value {
|
||||
None => Ok(default_schedule_task_type()),
|
||||
Some(CompatTaskType::Enum(task_type)) => Ok(task_type),
|
||||
Some(CompatTaskType::String(raw)) => {
|
||||
if raw.trim().is_empty() {
|
||||
Ok(default_schedule_task_type())
|
||||
} else if let Some(task_type) = parse_schedule_task_type(&raw) {
|
||||
Ok(task_type)
|
||||
} else {
|
||||
Err(serde::de::Error::custom(format!(
|
||||
"invalid schedule type '{raw}', expected one of: PlaylistUpdate, LibraryScan, GeoIpUpdate"
|
||||
)))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, Default, PartialEq, Eq)]
|
||||
@@ -10,8 +55,50 @@ pub enum ScheduleTaskType {
|
||||
pub struct ScheduleConfigDto {
|
||||
#[serde(default)]
|
||||
pub schedule: String,
|
||||
#[serde(default, rename = "type")]
|
||||
#[serde(
|
||||
default = "default_schedule_task_type",
|
||||
rename = "type",
|
||||
alias = "task_type",
|
||||
deserialize_with = "deserialize_schedule_task_type"
|
||||
)]
|
||||
pub task_type: ScheduleTaskType,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub targets: Option<Vec<String>>,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{ScheduleConfigDto, ScheduleTaskType};
|
||||
|
||||
#[test]
|
||||
fn defaults_to_playlist_update_when_type_is_missing() {
|
||||
let dto: ScheduleConfigDto = serde_saphyr::from_str("schedule: \"0 0 * * * * *\"\ntargets: [\"main\"]")
|
||||
.expect("schedule dto should parse");
|
||||
assert_eq!(dto.task_type, ScheduleTaskType::PlaylistUpdate);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn defaults_to_playlist_update_when_type_is_null_or_empty() {
|
||||
let null_type: ScheduleConfigDto =
|
||||
serde_saphyr::from_str("schedule: \"0 0 * * * * *\"\ntype: ~").expect("null type should parse");
|
||||
let empty_type: ScheduleConfigDto =
|
||||
serde_saphyr::from_str("schedule: \"0 0 * * * * *\"\ntype: \"\"").expect("empty type should parse");
|
||||
|
||||
assert_eq!(null_type.task_type, ScheduleTaskType::PlaylistUpdate);
|
||||
assert_eq!(empty_type.task_type, ScheduleTaskType::PlaylistUpdate);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accepts_legacy_task_type_aliases() {
|
||||
let legacy: ScheduleConfigDto = serde_saphyr::from_str("schedule: \"0 0 * * * * *\"\ntask_type: scan")
|
||||
.expect("legacy task_type alias should parse");
|
||||
assert_eq!(legacy.task_type, ScheduleTaskType::LibraryScan);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accepts_geoip_update_aliases() {
|
||||
let legacy: ScheduleConfigDto =
|
||||
serde_saphyr::from_str("schedule: \"0 0 * * * * *\"\ntype: geoip").expect("geoip alias should parse");
|
||||
assert_eq!(legacy.task_type, ScheduleTaskType::GeoIpUpdate);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
use crate::{
|
||||
model::DEFAULT_USER_AGENT,
|
||||
utils::{default_as_true, default_trakt_fuzzy_threshold, is_true, TRAKT_API_KEY, TRAKT_API_URL, TRAKT_API_VERSION},
|
||||
use crate::utils::{
|
||||
default_as_true, default_trakt_fuzzy_threshold, is_true, DEFAULT_USER_AGENT, TRAKT_API_KEY, TRAKT_API_URL,
|
||||
TRAKT_API_VERSION,
|
||||
};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::{fmt, str::FromStr};
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
use crate::{
|
||||
error::TuliproxError,
|
||||
info_err_res,
|
||||
model::DEFAULT_USER_AGENT,
|
||||
utils::{
|
||||
default_supported_video_extensions, is_blank_optional_str, is_blank_optional_string,
|
||||
is_default_supported_video_extensions, is_false,
|
||||
default_download_dir, default_episode_pattern, default_supported_video_extensions, is_blank_optional_str,
|
||||
is_blank_optional_string, is_blank_or_default_download_dir, is_blank_or_default_episode_pattern,
|
||||
is_default_supported_video_extensions, is_false, DEFAULT_USER_AGENT,
|
||||
},
|
||||
};
|
||||
use std::{borrow::BorrowMut, collections::HashMap};
|
||||
@@ -14,12 +14,12 @@ use std::{borrow::BorrowMut, collections::HashMap};
|
||||
pub struct VideoDownloadConfigDto {
|
||||
#[serde(default, skip_serializing_if = "HashMap::is_empty")]
|
||||
pub headers: HashMap<String, String>,
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
#[serde(default = "default_download_dir", skip_serializing_if = "is_blank_or_default_download_dir")]
|
||||
pub directory: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "is_false")]
|
||||
pub organize_into_directories: bool,
|
||||
// TODO use ptt
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
#[serde(default = "default_episode_pattern", skip_serializing_if = "is_blank_or_default_episode_pattern")]
|
||||
pub episode_pattern: Option<String>,
|
||||
}
|
||||
|
||||
@@ -27,8 +27,8 @@ impl VideoDownloadConfigDto {
|
||||
pub fn is_empty(&self) -> bool {
|
||||
!self.organize_into_directories
|
||||
&& self.headers.is_empty()
|
||||
&& is_blank_optional_str(self.directory.as_deref())
|
||||
&& is_blank_optional_str(self.episode_pattern.as_deref())
|
||||
&& is_blank_or_default_download_dir(&self.directory)
|
||||
&& is_blank_or_default_episode_pattern(&self.episode_pattern)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -69,11 +69,23 @@ impl VideoConfigDto {
|
||||
match &mut self.download {
|
||||
None => {}
|
||||
Some(downl) => {
|
||||
if is_blank_or_default_download_dir(&downl.directory) {
|
||||
downl.directory = default_download_dir();
|
||||
} else if let Some(directory) = downl.directory.as_ref() {
|
||||
downl.directory = Some(directory.trim().to_string());
|
||||
}
|
||||
|
||||
if downl.headers.is_empty() {
|
||||
downl.headers.borrow_mut().insert("Accept".to_string(), "video/*".to_string());
|
||||
downl.headers.borrow_mut().insert("User-Agent".to_string(), DEFAULT_USER_AGENT.to_string());
|
||||
}
|
||||
|
||||
if is_blank_or_default_episode_pattern(&downl.episode_pattern) {
|
||||
downl.episode_pattern = default_episode_pattern();
|
||||
} else if let Some(episode_pattern) = downl.episode_pattern.as_ref() {
|
||||
downl.episode_pattern = Some(episode_pattern.trim().to_string());
|
||||
}
|
||||
|
||||
if let Some(episode_pattern) = &downl.episode_pattern {
|
||||
if let Err(err) = crate::model::REGEX_CACHE.get_or_compile(episode_pattern) {
|
||||
return info_err_res!("can't parse regex: {episode_pattern} {err}");
|
||||
@@ -84,3 +96,102 @@ impl VideoConfigDto {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::utils::DEFAULT_DOWNLOAD_DIR;
|
||||
|
||||
#[test]
|
||||
fn prepare_sets_default_download_dir_when_missing() {
|
||||
let mut video = VideoConfigDto {
|
||||
extensions: Vec::new(),
|
||||
download: Some(VideoDownloadConfigDto {
|
||||
headers: HashMap::new(),
|
||||
directory: None,
|
||||
organize_into_directories: false,
|
||||
episode_pattern: None,
|
||||
}),
|
||||
web_search: None,
|
||||
};
|
||||
video.prepare().expect("prepare should succeed");
|
||||
let download = video.download.expect("download should exist");
|
||||
assert_eq!(download.directory.as_deref(), Some(DEFAULT_DOWNLOAD_DIR));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prepare_sets_default_episode_pattern_when_missing() {
|
||||
let mut video = VideoConfigDto {
|
||||
extensions: Vec::new(),
|
||||
download: Some(VideoDownloadConfigDto {
|
||||
headers: HashMap::new(),
|
||||
directory: None,
|
||||
organize_into_directories: false,
|
||||
episode_pattern: None,
|
||||
}),
|
||||
web_search: None,
|
||||
};
|
||||
video.prepare().expect("prepare should succeed");
|
||||
let download = video.download.expect("download should exist");
|
||||
assert!(download.episode_pattern.is_some(), "expected default episode pattern to be set");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn prepare_keeps_custom_download_dir() {
|
||||
let mut video = VideoConfigDto {
|
||||
extensions: Vec::new(),
|
||||
download: Some(VideoDownloadConfigDto {
|
||||
headers: HashMap::new(),
|
||||
directory: Some("custom-downloads".to_string()),
|
||||
organize_into_directories: false,
|
||||
episode_pattern: None,
|
||||
}),
|
||||
web_search: None,
|
||||
};
|
||||
video.prepare().expect("prepare should succeed");
|
||||
let download = video.download.expect("download should exist");
|
||||
assert_eq!(download.directory.as_deref(), Some("custom-downloads"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serializing_skips_default_download_dir() {
|
||||
let download = VideoDownloadConfigDto {
|
||||
headers: HashMap::new(),
|
||||
directory: Some(DEFAULT_DOWNLOAD_DIR.to_string()),
|
||||
organize_into_directories: false,
|
||||
episode_pattern: None,
|
||||
};
|
||||
let serialized = serde_json::to_string(&download).expect("download serialization should succeed");
|
||||
assert!(
|
||||
!serialized.contains("\"directory\""),
|
||||
"expected no directory field for default value, got: {serialized}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serializing_keeps_custom_download_dir() {
|
||||
let download = VideoDownloadConfigDto {
|
||||
headers: HashMap::new(),
|
||||
directory: Some("custom-downloads".to_string()),
|
||||
organize_into_directories: false,
|
||||
episode_pattern: None,
|
||||
};
|
||||
let serialized = serde_json::to_string(&download).expect("download serialization should succeed");
|
||||
assert!(serialized.contains("\"directory\""), "expected directory field for custom value, got: {serialized}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serializing_skips_default_episode_pattern() {
|
||||
let download = VideoDownloadConfigDto {
|
||||
headers: HashMap::new(),
|
||||
directory: None,
|
||||
organize_into_directories: false,
|
||||
episode_pattern: default_episode_pattern(),
|
||||
};
|
||||
let serialized = serde_json::to_string(&download).expect("download serialization should succeed");
|
||||
assert!(
|
||||
!serialized.contains("\"episode_pattern\""),
|
||||
"expected no episode_pattern field for default value, got: {serialized}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
use crate::utils::{
|
||||
default_as_true, default_token_ttl_mins, is_blank_optional_str, is_blank_optional_string,
|
||||
default_as_true, default_token_ttl_mins, default_user_file_path, is_blank_or_default_user_file_path,
|
||||
is_default_token_ttl_mins, is_true,
|
||||
};
|
||||
|
||||
@@ -12,7 +12,7 @@ pub struct WebAuthConfigDto {
|
||||
pub secret: String,
|
||||
#[serde(default = "default_token_ttl_mins", skip_serializing_if = "is_default_token_ttl_mins")]
|
||||
pub token_ttl_mins: u32,
|
||||
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
|
||||
#[serde(default = "default_user_file_path", skip_serializing_if = "is_blank_or_default_user_file_path")]
|
||||
pub userfile: Option<String>,
|
||||
}
|
||||
|
||||
@@ -35,6 +35,6 @@ impl WebAuthConfigDto {
|
||||
&& self.token_ttl_mins == empty.token_ttl_mins
|
||||
&& self.issuer.trim().is_empty()
|
||||
&& self.secret.trim().is_empty()
|
||||
&& is_blank_optional_str(self.userfile.as_deref())
|
||||
&& is_blank_or_default_user_file_path(&self.userfile)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,6 +7,14 @@ use std::{
|
||||
|
||||
pub const DEFAULT_HOME_ENV_VAR: &str = "TULIPROX_HOME";
|
||||
pub const DEFAULT_WEB_ROOT_ENV_VAR: &str = "TULIPROX_WEB_ROOT";
|
||||
pub const DEFAULT_USER_AGENT: &str = "VLC/3.0.16 LibVLC/3.0.16";
|
||||
pub const DEFAULT_BACKUP_DIR: &str = "backup";
|
||||
pub const DEFAULT_CACHE_DIR: &str = "cache";
|
||||
pub const DEFAULT_STORAGE_TEMP_DIR: &str = "tmp";
|
||||
pub const DEFAULT_USER_CONFIG_DIR: &str = "user_config";
|
||||
pub const DEFAULT_DOWNLOAD_DIR: &str = "downloads";
|
||||
pub const DEFAULT_CUSTOM_STREAM_RESPONSE_PATH: &str = "resources";
|
||||
pub const DNS_RESOLVED_FILE: &str = "provider_dns_resolved.json";
|
||||
pub const USER_FILE: &str = "user.txt";
|
||||
pub const CONFIG_PATH: &str = "config";
|
||||
pub const CONFIG_FILE: &str = "config.yml";
|
||||
@@ -15,7 +23,7 @@ pub const MAPPING_FILE: &str = "mapping.yml";
|
||||
pub const TEMPLATE_FILE: &str = "template.yml";
|
||||
pub const API_PROXY_FILE: &str = "api-proxy.yml";
|
||||
pub const DEFAULT_WEB_DIR: &str = "web";
|
||||
pub const DEFAULT_WORKING_DIR: &str = "data";
|
||||
pub const DEFAULT_STORAGE_DIR: &str = "data"; // TODO rename to storage and use data for config, storage, ...
|
||||
pub const DEFAULT_PORT: u16 = 8901;
|
||||
pub const ENCODING_GZIP: &str = "gzip";
|
||||
pub const ENCODING_DEFLATE: &str = "deflate";
|
||||
@@ -31,6 +39,8 @@ pub const HLS_EXT_FRAGMENT: &str = ".m3u8#";
|
||||
pub const DASH_EXT_QUERY: &str = ".mpd?";
|
||||
pub const DASH_EXT_FRAGMENT: &str = ".mpd#";
|
||||
|
||||
pub const DEFAULT_EPISODE_PATTERN: &str = r".*(?P<episode>[Ss]\d{1,2}(.*?)[Ee]\d{1,2}).*";
|
||||
|
||||
pub const FILENAME_TRIM_PATTERNS: &[char] = &['.', '-', '_'];
|
||||
|
||||
const SUPPORTED_RESPONSE_HEADERS: &[&str] = &[
|
||||
|
||||
@@ -1,4 +1,11 @@
|
||||
use crate::model::{ConfigTargetOptions, LibraryMetadataFormat, ProcessingOrder};
|
||||
use crate::{
|
||||
model::{ConfigTargetOptions, LibraryMetadataFormat, MetadataUpdateConfigDto, ProcessingOrder, VideoConfigDto},
|
||||
utils::{
|
||||
CONFIG_PATH, DEFAULT_BACKUP_DIR, DEFAULT_CACHE_DIR, DEFAULT_CUSTOM_STREAM_RESPONSE_PATH, DEFAULT_DOWNLOAD_DIR,
|
||||
DEFAULT_EPISODE_PATTERN, DEFAULT_STORAGE_DIR, DEFAULT_USER_AGENT, DEFAULT_USER_CONFIG_DIR, DEFAULT_WEB_DIR,
|
||||
MAPPING_FILE, TEMPLATE_FILE, USER_FILE,
|
||||
},
|
||||
};
|
||||
use std::sync::Arc;
|
||||
|
||||
pub const fn is_zero_u16(v: &u16) -> bool { *v == 0 }
|
||||
@@ -218,6 +225,112 @@ pub fn is_default_metadata_ffprobe_live_probe_size(v: &String) -> bool {
|
||||
*v == default_metadata_ffprobe_live_probe_size()
|
||||
}
|
||||
|
||||
pub fn get_default_web_root() -> String { DEFAULT_WEB_DIR.to_string() }
|
||||
pub fn is_blank_or_default_web_root(value: &str) -> bool {
|
||||
let normalized = value.trim().replace('\\', "/");
|
||||
if normalized.is_empty() {
|
||||
return true;
|
||||
}
|
||||
|
||||
let normalized = normalized.trim_end_matches('/');
|
||||
normalized.trim_start_matches("./") == DEFAULT_WEB_DIR
|
||||
}
|
||||
|
||||
pub fn is_default_dir_path(value: &str, default_dir: &str) -> bool {
|
||||
let normalized = value.trim().replace('\\', "/");
|
||||
let normalized = normalized.trim_end_matches('/');
|
||||
let normalized = normalized.trim_start_matches("./");
|
||||
let default_dir = default_dir.trim().replace('\\', "/");
|
||||
let default_dir = default_dir.trim_end_matches('/');
|
||||
normalized == default_dir
|
||||
}
|
||||
|
||||
pub fn is_blank_or_default_download_dir(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || is_default_dir_path(value, DEFAULT_DOWNLOAD_DIR))
|
||||
}
|
||||
pub fn default_download_dir() -> Option<String> { Some(DEFAULT_DOWNLOAD_DIR.to_string()) }
|
||||
|
||||
pub fn default_episode_pattern() -> Option<String> { Some(DEFAULT_EPISODE_PATTERN.to_string()) }
|
||||
|
||||
pub fn is_blank_or_default_episode_pattern(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || value.trim() == DEFAULT_EPISODE_PATTERN)
|
||||
}
|
||||
|
||||
pub fn is_blank_or_default_cache_dir(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || is_default_dir_path(value, DEFAULT_CACHE_DIR))
|
||||
}
|
||||
|
||||
pub fn default_default_user_agent() -> Option<String> { Some(DEFAULT_USER_AGENT.to_string()) }
|
||||
pub fn default_main_storage_dir() -> Option<String> { Some(DEFAULT_STORAGE_DIR.to_string()) }
|
||||
pub fn default_main_backup_dir() -> Option<String> { Some(DEFAULT_BACKUP_DIR.to_string()) }
|
||||
pub fn default_main_user_config_dir() -> Option<String> { Some(DEFAULT_USER_CONFIG_DIR.to_string()) }
|
||||
pub fn default_main_mapping_path() -> Option<String> { Some(format!("./{CONFIG_PATH}/{MAPPING_FILE}")) }
|
||||
pub fn default_main_template_path() -> Option<String> { Some(format!("./{CONFIG_PATH}/{TEMPLATE_FILE}")) }
|
||||
pub fn default_custom_stream_response_path() -> Option<String> { Some(DEFAULT_CUSTOM_STREAM_RESPONSE_PATH.to_string()) }
|
||||
pub fn default_user_file_path() -> Option<String> { Some(format!("./{CONFIG_PATH}/{USER_FILE}")) }
|
||||
fn is_default_config_file_path(value: &str, file_name: &str) -> bool {
|
||||
let normalized = value.trim().replace('\\', "/");
|
||||
let normalized = normalized.trim_start_matches("./");
|
||||
normalized == file_name
|
||||
|| normalized.rsplit_once('/').is_some_and(|(dir, file)| dir == CONFIG_PATH && file == file_name)
|
||||
}
|
||||
|
||||
pub fn is_blank_or_default_custom_stream_response_path(path: &Option<String>) -> bool {
|
||||
path.as_ref()
|
||||
.is_none_or(|value| value.trim().is_empty() || is_default_dir_path(value, DEFAULT_CUSTOM_STREAM_RESPONSE_PATH))
|
||||
}
|
||||
|
||||
pub fn is_blank_or_default_mapping_path(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || is_default_config_file_path(value, MAPPING_FILE))
|
||||
}
|
||||
|
||||
pub fn is_blank_or_default_template_path(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || is_default_config_file_path(value, TEMPLATE_FILE))
|
||||
}
|
||||
|
||||
pub fn is_blank_or_default_storage_dir(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || is_default_dir_path(value, DEFAULT_STORAGE_DIR))
|
||||
}
|
||||
|
||||
pub fn is_blank_or_default_backup_dir(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || is_default_dir_path(value, DEFAULT_BACKUP_DIR))
|
||||
}
|
||||
|
||||
pub fn is_blank_or_default_user_config_dir(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || is_default_dir_path(value, DEFAULT_USER_CONFIG_DIR))
|
||||
}
|
||||
|
||||
pub fn is_blank_or_default_user_file_path(path: &Option<String>) -> bool {
|
||||
path.as_ref().is_none_or(|value| value.trim().is_empty() || is_default_config_file_path(value, USER_FILE))
|
||||
}
|
||||
|
||||
pub fn normalize_optional_dir(path: &Option<String>, default_dir: &str) -> Option<String> {
|
||||
path.as_ref().and_then(|value| {
|
||||
if value.trim().is_empty() || is_default_dir_path(value, default_dir) {
|
||||
None
|
||||
} else {
|
||||
Some(value.clone())
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
pub fn normalize_optional_config_file_path(path: &Option<String>, default_file_name: &str) -> Option<String> {
|
||||
path.as_ref().and_then(|value| {
|
||||
if value.trim().is_empty() || is_default_config_file_path(value, default_file_name) {
|
||||
None
|
||||
} else {
|
||||
Some(value.clone())
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
pub fn is_none_or_empty_video(video: &Option<VideoConfigDto>) -> bool {
|
||||
video.as_ref().is_none_or(VideoConfigDto::is_empty)
|
||||
}
|
||||
pub fn is_none_or_empty_metadata_update(metadata_update: &Option<MetadataUpdateConfigDto>) -> bool {
|
||||
metadata_update.as_ref().is_none_or(MetadataUpdateConfigDto::is_empty)
|
||||
}
|
||||
|
||||
//////////////////////////////
|
||||
// HDHomerun Device Defaults
|
||||
//////////////////////////////
|
||||
|
||||
@@ -8,6 +8,7 @@ use std::{borrow::Cow, sync::atomic::Ordering};
|
||||
use url::Url;
|
||||
|
||||
pub const PROVIDER_SCHEME_PREFIX: &str = "provider://";
|
||||
pub const BATCH_SCHEME_PREFIX: &str = "batch://";
|
||||
|
||||
pub const CONTENT_TYPE_JSON: &str = "application/json";
|
||||
pub const CONTENT_TYPE_CBOR: &str = "application/cbor";
|
||||
|
||||
Reference in New Issue
Block a user