euzu/tuliprox
1
0
Fork 0
mirror of https://github.com/euzu/tuliprox.git synced 2026-10-31 11:12:36 +01:00
Code Issues Packages Projects Releases 5 Wiki Activity

Feature/stream probe (#585)

Browse Source 
Stream probe
This commit is contained in:
euzu
2026-02-14 17:52:35 +01:00
committed by GitHub
parent 9b760a5509
commit 7b8bdaa49b
55 changed files with 2629 additions and 1917 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/ci.yaml
+2 -2
View File
@@ -1,8 +1,8 @@
name: CI
on:
push:
branches: [develop]
# push:
# branches: []
pull_request:
branches: [develop]
CHANGELOG.md
+189 -120
View File
@@ -1,11 +1,11 @@
# Changelog
## 3.3.0 (2026-01-03)
## Unreleased
### ⚠️ Breaking Changes
## ⚠️ Breaking Changes
- **Global Input Definitions**: To align input definitions with the SourceEditor, inputs are now defined globally in the `inputs` section of the config
file. Each source can reference one or more inputs by their name in the `inputs` attribute.
- **Global Input Definitions**: To align input definitions with the SourceEditor, inputs are now defined globally in the `inputs` section of the
config file. Each source can reference one or more inputs by their name in the `inputs` attribute.
- **Data Format Migration**: Due to heavy refactoring, the old data format is invalid. You need to clean your `data` folder and update the playlists.
- **B+Tree Storage Format**: Storage format has changed to a more efficient Slotted Page architecture.
- **Index optimization**: Added index to B+Tree to accelerate queries without tree traversal.
@@ -20,7 +20,7 @@
- **Favorites Redesign**: Replaced implicit `create_alias` with explicit `add_favourite(group_name)` script function.
- **EpgSmartMatch**: Field `name_prefix` syntax needs to be changed from `name_prefix: !suffix "."` to `name_prefix: { suffix: "." }`.
- **Sort**: Sort can now use filter to sort specific entries.
```yaml
sort:
match_as_ascii: true
@@ -37,12 +37,14 @@
- "!CHAN_SEQ!"
- '(?i)\bHD\b'
- '(?i)\bSD\b'
```
```
- Trakt api config field `key` is now `api_key`. Added `user_agent` field to Trakt api config
- resolve_vod_delay and resolve_series_delay are now merged as resolve_delay, added `probe_live` and `probe_live_interval_hours` for live stream probing.
- resolve_vod_delay and resolve_series_delay are now merged as resolve_delay, added `probe_live` and `probe_live_interval_hours` for live stream
probing.
```yaml
# Before (deprecated)
output:
- type: xtream
@@ -51,8 +53,9 @@
resolve_series: true
resolve_series_delay: 2
```
```yaml
# After (new consolidated)
output:
- type: xtream
@@ -61,10 +64,10 @@
resolve_delay: 2 # Single delay for all resolution types
```
### 🌟 New Features
## 🌟 New Features
- **Smart Connection Priority**: Introduced a priority system for connections. Users with higher priority can preempt (kick) lower priority connections
(e.g., background tasks or standard users) when provider slots are full.
- **Smart Connection Priority**: Introduced a priority system for connections. Users with higher priority can preempt (kick) lower priority
connections (e.g., background tasks or standard users) when provider slots are full.
- **Background Metadata Queue**: Metadata resolution (VOD/Series) and stream analysis are now queued per input and processed in the background when
provider connections are idle. This prevents "No Connections" errors for active users during playlist updates.
- **Live TV Probing**: Added support for probing Live TV streams (`probe_live`) to determine codecs and resolution. This runs as a low-priority
@@ -75,10 +78,11 @@
- Recursive scanning, automatic classification, and NFO/TMDB metadata resolution.
- Incremental scanning and virtual ID management.
- **Panel API Integration**: Optional integration to renew expired input accounts or provision new accounts to ensure a minimum valid input accounts.
- **Playlist Caching**: Added `cache_duration` to inputs, allowing configurable provider playlist cache times during subsequent updates
(e.g., `60s`, `5m` `12h`, `1d`).
- **Playlist Caching**: Added `cache_duration` to inputs, allowing configurable provider playlist cache times during subsequent updates (e.g., `60s`,
`5m` `12h`, `1d`).
- **Database Viewer**: New CLI flags `--dbx` and `--dbm` to inspect internal database content.
- **Added `disk_based_processing`**: (boolean, default `false`) to `config.yml`. When enabled, input playlists are processed from disk instead of memory.
- **Added `disk_based_processing`**: (boolean, default `false`) to `config.yml`. When enabled, input playlists are processed from disk instead of
memory.
- **User-Agent `default_user_agent`**: Ensures that outgoing requests always pass a default user agent.
- **FFprobe Integration**: Added capability to probe streams for codec, resolution, HDR (HDR10/HLG/DV), and audio channels using `ffprobe`. Probing
strictly respects provider connection limits. If no slot is available (considering user limits), the item is skipped to prevent provider bans.
@@ -86,13 +90,13 @@
- **Streaming**: Added `grace_period_hold_stream` configuration option to delay stream output until grace period connection checks are completed.
- **Provider Failover & Rotation**: Tuliprox supports robust failover mechanisms for streaming providers.
You can use the special `provider://<provider_name>/...` URL scheme in your configurations. Tuliprox will automatically resolve this to the current
active URL of the specified provider.
active URL of the specified provider.
If the current URL fails (e.g., 5xx error, timeout), Tuliprox automatically rotates to the next available URL for that provider.
It tracks failures and prevents infinite loops by limiting attempts to the number of available URLs.
- Added `epg_request_timeshift: [-+]hh:mm or TimeZone`, example `Europe/Paris`, `America/New_York`, `-2:30`(-2h30m), `+0:15` (15m), `2` (2h),
`:30` (30m), `:3` (3m)
- Added `epg_request_timeshift: [-+]hh:mm or TimeZone`, example `Europe/Paris`, `America/New_York`, `-2:30`(-2h30m), `+0:15` (15m), `2` (2h), `:30`
(30m), `:3` (3m)
### ⚙️ New Settings
## ⚙️ New Settings
- **config.yml**:
- Added `video.ffprobe_enabled` (default: false) and `video.ffprobe_timeout`.
@@ -104,13 +108,13 @@
- Added `probe_live_interval_hours`: Sets the frequency for re-probing Live TV streams.
- Added `resolve_background`: Toggles background metadata resolution (default `true`). Set to `false` for blocking, immediate resolution.
### 🛠 Optimizations
## 🛠 Optimizations
- **Quality Tagging**: Generates enhanced filename tags (e.g., `[2160p 4K HEVC HDR TrueHD 7.1]`) for STRM files based on analysis results.
- **Flat Grouping**: When `flat: true` option for STRM output is active, multiple versions (e.g., 4K and 1080p) of the same movie are now safely
merged over all categories into a single folder based on TMDB ID, compatible with Jellyfin/Emby "Multi-Version" features.
### ⚙️ Engine & Storage Optimizations
## ⚙️ Engine & Storage Optimizations
- **Slotted Page Architecture**: Improved space utilization and support for variable-length keys.
- **Adaptive LZ4 Compression**: Optimized disk footprint for stored values.
@@ -128,16 +132,18 @@
- **String Interning**: Implemented `Arc<str>` string interning for playlist items to further reduce memory footprint.
- **Zero-Copy B+Tree Scan**: Implemented zero-copy scanning for B+Tree internal nodes, significantly reducing heap allocations and improving random
read throughput (up to 96k ops/sec).
- **Optimized Key Lookups**: `XtreamRepository` and `M3uRepository` now use zero-copy queries for `u32` keys, enhancing performance for high-traffic endpoints.
- **Optimized Key Lookups**: `XtreamRepository` and `M3uRepository` now use zero-copy queries for `u32` keys, enhancing performance for high-traffic
endpoints.
### 🔍 Mapping & Filtering Enhancements
## 🔍 Mapping & Filtering Enhancements
- **Accent-Independent Matching**: Integrated `match_as_ascii` flag for robust text matching (e.g., "Cinema" matches "Cinéma").
- **Deunicoding Support**: `ValueProvider` and `ValueAccessor` now support on-the-fly deunicoding.
- **Flexible Sorting**: Added `order: none` support to retain source order in mappings.
- **Mapper Loop enhancement**: Updated `for_each` syntax to `variable.for_each((key, value) => { ... })`. Added support for `_` ignored variables in loop.
- **Mapper Loop enhancement**: Updated `for_each` syntax to `variable.for_each((key, value) => { ... })`. Added support for `_` ignored variables in
loop.
### 💻 WebUI & API
## 💻 WebUI & API
- **Source Editor Integration**: Redesigned UI for global input management and hot-reloading.
- **Messaging Config View**: New UI for configuring Discord and enhanced REST settings.
@@ -149,7 +155,7 @@
- Added **Stream Buffer** settings (Enabled, Size) to Reverse Proxy configuration UI.
- Added **TMDB** settings (Rate Limit, Cache Duration, Language) and **Metadata Formats** (NFO support) to Library configuration UI.
### 🚀 Performance & Stability
## 🚀 Performance & Stability
- **Deadlock Resolution**: Fixed a potential deadlock in `ProviderLineupManager::reconcile_connections` by refactoring `DashMap` iterations to use
snapshots, preventing internal shard locks from being held during async lock acquisition.
@@ -187,7 +193,7 @@
- **EPG**: Fixed XMLTV timeshift to correctly apply user-defined timezone offsets in the generated XML output.
- **407 Proxy Authentication Required** fix.
### ⚙️ Messaging Refactoring
## ⚙️ Messaging Refactoring
- **Structured Messaging**: Transitioned from JSON-string-based notifications to a strictly typed messaging pipeline.
- **Backend Model Migration**: Moved complex messaging models (`WatchChanges`, `ProcessingStats`) from the shared crate to the backend to reduce
@@ -208,7 +214,8 @@
- Fixed SSL certificate field binding in configuration view
- More robust connection-state and provider-handle management
- Streamlined event notifications and provider-count reporting
- Added configurable `reverse_proxy.resource_retry` (UI + server) to tune max attempts, base delay, and exponential backoff multiplier for proxied resources.
- Added configurable `reverse_proxy.resource_retry` (UI + server) to tune max attempts, base delay, and exponential backoff multiplier for proxied
resources.
- Multi Strm outputs with same type is now allowed.
- Added new mapper function `pad(text | number, number, char, optional position: "<" | ">" | "^")`
- Added new mapper function `format` for simple in-text replacement like `format("Hello {}! Hello {}!", "Bob", "World")`
@@ -229,7 +236,9 @@
- Added `use_memory_cache` attribute to target definition to hold playlist in memory to reduce disc access.
Placing playlist into memory causes more RAM usage but reduces disk access.
- Added optional `filter` attribute to Output (except HDHomerun-Output).
Output filters are applied after all transformations have been performed, therefore, all filter contents must refer to the final state of the playlist.
Output filters are applied after all transformations have been performed, therefore, all filter contents must refer to the final state of the
playlist.
- Added burst buffer to shared stream
- Telegram message thread support. thread id can now be appended to chat-id like `chat-id:thread-id`.
- Telegram supports markdown generation for structured json messages. simply set `markdown: true` in telegram config.
@@ -238,8 +247,9 @@
- Added standardized SSDP (Simple Service Discovery Protocol) and the Proprietary HDHomeRun UDP Discovery Protocol (Port 65001)
- Fixed some session handling issue
- added `reverse_proxy.disabled_header` configuration
Allows removing selected headers before forwarding requests when acting as a reverse proxy. Configure removal of the referer header,
all `X-*` headers, and additional custom headers.
Allows removing selected headers before forwarding requests when acting as a reverse proxy. Configure removal of the referer header, all `X-*`
headers, and additional custom headers.
- !BREAKING_CHANGE! `disble_referer_header` is now part of `reverse_proxy.disabled_header` configuration
- UserTable: Copy credentials to clipboard from user table
- UserTable: Kick user action from streams table
@@ -259,7 +269,8 @@
- WebSocket now reconnects on disconnect; added WebSocket connection status icon in Web UI
- Added Playlist EPG view with timeline, channels, `now` line, and program details
- EPG data can now be fetched from selected targets and custom URLs
- Faster, more reliable EPG loading via streaming and asynchronous processing, with reduced memory usage and better support for large or compressed guides.
- Faster, more reliable EPG loading via streaming and asynchronous processing, with reduced memory usage and better support for large or compressed
guides.
- Invalid EPG text data fix
- Added new sidebar entry and icon for quick EPG access
- Added CBOR (binary JSON) support for large API data
@@ -295,9 +306,11 @@
- added sleep timer `sleep_timer_mins` to config.yml
- added mapper script builtin function `template` to access template definitions.
```toml
station_prefix = template(concat("US_", station, "_PREFIX")),
```
```text
station_prefix = template(concat("US_", station, "_PREFIX")),
```
If we assume the variable `station` contains the value `WINK`,
this script receives the template with the concatenated name `US_WINK_PREFIX` which should be defined in `templates` section,
@@ -308,7 +321,7 @@ and assigns it to the variable `station_prefix`.
- Optional flat directory structure via 'flat' parameter (nested folder structures are not supported by some media scanners).
- Added Trakt support for XC targets
```yaml
```yaml
- name: iptv-trakt-example
output:
- type: xtream
@@ -332,7 +345,7 @@ and assigns it to the variable `station_prefix`.
category_name: "📺 Latest TV Shows"
content_type: "series"
fuzzy_match_threshold: 80
```
```
## 3.1.3 (2025-06-06)
@@ -341,21 +354,23 @@ and assigns it to the variable `station_prefix`.
- Mapper script can have blocks now.
For example, you want to write a `if then else` block
```txt
# Maybe there is no station
station = @Caption ~ "(ABC)"
match {
station => {
# if block
# station exists
}
# optional any match as else block
_ => {
# else block
# station does not exists
}
}
```
```text
# Maybe there is no station
station = @Caption ~ "(ABC)"
match {
station => {
# if block
# station exists
}
# optional any match as else block
_ => {
# else block
# station does not exists
}
}
```
- New BuiltIn Mapper function `first`. When you use Regular expressions it could be that your match contains multiple results.
The builtin function `first` returns the first match.
@@ -384,7 +399,6 @@ and assigns it to the variable `station_prefix`.
- channel_unavailable.ts
`user_account_expired.ts`: Tuliprox will return a 403 Forbidden response for any playlist request if the user is expired.
So this screen will only ever appear if someone tries to directly access a stream URL after their account has expired.
- !BREAKING_CHANGE! epg refactored
- url config is now renamed to sources
- Added `priority`, priority is `optional`
@@ -424,9 +438,10 @@ epg:
You can now place multiple mapping files inside a directory like `mapping.d` and specify it using the `-m` option, for example:
`-m /home/tuliprox/config/mapping.d`
The files are loaded in **alphanumeric** order.
**Note:** This is a lexicographic sort — so `m_10.yml` comes before `m_2.yml` unless you name files carefully (e.g., `m_01.yml`, `m_02.yml`, ..., `m_10.yml`).
- Added `mapping_path` to `config.yml`.
**Note:** This is a lexicographic sort — so `m_10.yml` comes before `m_2.yml` unless you name files carefully (e.g., `m_01.yml`, `m_02.yml`, ...,
`m_10.yml`).
- Added `mapping_path` to `config.yml`.
- Added list template for sequences. List templates can only be used for sequences.
```yaml
@@ -455,10 +470,11 @@ The template can now be used for sequence
- added `disable_referer_header` to `reverse_proxy` config
This option, when set to `true`, prevents tuliprox from sending the Referer header in requests made when acting as a reverse proxy. This can be
particularly useful when dealing with certain Xtream Codes providers that might restrict or behave differently based on the Referer header.
Default is `false`.
particularly useful when dealing with certain Xtream Codes providers that might restrict or behave differently based on the Referer header. Default is
`false`.
```yaml
reverse_proxy:
disable_referer_header: false
```
@@ -476,6 +492,7 @@ reverse_proxy:
- !BREAKING_CHANGE! `web_ui config` restructured and added `user_ui_enabled` attribute
```yaml
web_ui:
enabled: true
user_ui_enabled: true
@@ -485,6 +502,7 @@ web_ui:
issuer: tuliprox
secret: ef9ab256a8c0abe5de92c2e05ca92baa810472ab702ff1674e9248308ceeec92
userfile: user.txt
```
- `grace_period_millis` default set to 300 milliseconds.
@@ -500,11 +518,13 @@ web_ui:
The configuration below allows up to 10 connections initially and then replenishes 1 connection every 500 milliseconds.
```yaml
reverse_proxy:
rate_limit:
enabled: true
period_millis: 500
burst_size: 10
```
- Multi epg processing/optimization, auto guessing/assigning epg id's
@@ -512,10 +532,13 @@ reverse_proxy:
- Added `force_redirect` to target config options. valid options are `live`, `vod`, `series`
```yaml
options: {ignore_logo: false, share_live_streams: false, force_redirect: [vod, series]}
```
```yaml
epg:
url: ['http://localhost:3001/xmltv.php?epg_id=1', 'http://localhost:3001/xmltv.php?epg_id=2']
smart_match:
@@ -527,6 +550,7 @@ epg:
name_prefix_separator: [':', '|', '-']
strip : ["3840p", "uhd", "fhd", "hd", "sd", "4k", "plus", "raw"]
normalize_regex: '[^a-zA-Z0-9\-]'
```
`match_threshold`is optional and if not set 80.
@@ -536,18 +560,27 @@ epg:
`normalize_regex: [^a-zA-Z0-9\-]` is the default
```yaml
# single epg
url: 'https://localhost.com/epg.xml'
```
```yaml
# multi local file epg
url: ['file:///${env:TULIPROX_HOME}/epg.xml', 'file:///${env:TULIPROX_HOME}/epg2.xml']
```
```yaml
# multi url epg
url: ['http://localhost:3001/xmltv.php?epg_id=1', 'http://localhost:3001/xmltv.php?epg_id=2']
```
- Added `strip` to input for auto epg matching, if not given `["3840p", "uhd", "fhd", "hd", "sd", "4k", "plus", "raw"]` is default
@@ -563,13 +596,15 @@ url: ['http://localhost:3001/xmltv.php?epg_id=1', 'http://localhost:3001/xmltv.p
- hdhomerun supports now basic auth like <http://user:password@ip:port/lineup.json>
you need to enable auth in config
```yaml
hdhomerun:
enabled: true
auth: true
devices:
- name: hdhr1
```
```yaml
hdhomerun:
enabled: true
auth: true
devices:
- name: hdhr1
```
- A new filter field `caption` has been added. This field is used to bypass the `title/name` issue.
If `caption` is provided, its value is read from `title` if available, otherwise from `name`.
@@ -577,35 +612,37 @@ url: ['http://localhost:3001/xmltv.php?epg_id=1', 'http://localhost:3001/xmltv.p
- Counter has now an attribute `padding`. Which fills the number like 001.
- Added proxy configuration for all outgoing requests in `config.yml`. supported http, https, socks5 proxies.
```yaml
proxy:
url: socks5://192.168.1.6:8123
username: uname # <- optional basic auth
password: secret # <- optional basic auth
```
```yaml
proxy:
url: socks5://192.168.1.6:8123
username: uname # <- optional basic auth
password: secret # <- optional basic auth
```
- Added support for regular expression-based sequence sorting.
You can now sort both groups and channels using custom regex sequences.
```yaml
sort:
groups:
order: asc
sequence:
- '^Freetv'
- '^Shopping'
- '^Entertainment'
- '^Sunrise'
channels:
- field: caption
group_pattern: '^Freetv'
order: asc
sequence:
- '(?P<c1>.*?)\bUHD\b'
- '(?P<c1>.*?)\bFHD\b'
- '(?P<c1>.*?)\bHD\b'
- '(?P<c1>.*?)\bSD\b'
```
```yaml
sort:
groups:
order: asc
sequence:
- '^Freetv'
- '^Shopping'
- '^Entertainment'
- '^Sunrise'
channels:
- field: caption
group_pattern: '^Freetv'
order: asc
sequence:
- '(?P<c1>.*?)\bUHD\b'
- '(?P<c1>.*?)\bFHD\b'
- '(?P<c1>.*?)\bHD\b'
- '(?P<c1>.*?)\bSD\b'
```
In the example above, groups are sorted based on the specified sequence.
Channels within the `Freetv` group are first sorted by `quality` (as matched by the regex sequence), and then by the `captured prefix`.
@@ -621,8 +658,10 @@ The numeric suffix indicates the priority: `c1` is evaluated first, followed by
- pattern_ipv6 # Optional regex pattern to extract IPv6
```yaml
ipcheck:
url_ipv4: https://ipinfo.io/ip
```
## 2.2.5 (2025-03-27)
@@ -655,20 +694,22 @@ ipcheck:
- hls reverse proxy implemented
- dash redirect implemented (reverse proxy not supported)
- !BREAKING CHANGE! `channel_unavailable_file` is now under `custom_stream_response`,
- New custom streams `user_connections_exhausted` and `provider_connections_exhausted` added.
- New custom streams `user_connections_exhausted` and `provider_connections_exhausted`added.
```yaml
custom_stream_response:
channel_unavailable: /home/tuliprox/resources/channel_unavailable.ts
user_connections_exhausted: /home/tuliprox/resources/user_connections_exhausted.ts
provider_connections_exhausted: /home/tuliprox/resources/provider_connections_exhausted.ts
```
- input alias definition for same provider with same content but different credentials
```yaml
- sources:
- inputs:
- inputs:
- type: xtream
name: my_provider
url: 'http://provider.net'
@@ -679,8 +720,8 @@ custom_stream_response:
url: 'http://provider.net'
username: abcd
password: secret2
targets:
- name: test
targets:
- name: test
```
Input aliases can be defined as batches in csv files with `;` separator.
@@ -689,34 +730,38 @@ There are 2 batch input types `xtream_batch` and `m3u_batch`.
```yaml
- sources:
- inputs:
- inputs:
- type: xtream_batch
url: 'file:///home/tuliprox/config/my_provider_batch.csv'
targets:
- name: test
targets:
- name: test
```
```csv
#name;username;password;url;max_connections;priority
my_provider_1;user1;password1;http://my_provider_1.com:80;1;0
my_provider_2;user2;password2;http://my_provider_2.com:8080;1;0
```
`M3uBatch`:
```yaml
- sources:
- inputs:
- inputs:
- type: m3u_batch
url: 'file:///home/tuliprox/config/my_provider_batch.csv'
targets:
targets:
- name: test
```
```csv
#url;max_connections;priority
http://my_provider_1.com:80/get_php?username=user1&password=password1;1;0
http://my_provider_2.com:8080/get_php?username=user2&password=password2;1;0
```
The Fields `max_connections` and `priority`are optional.
@@ -786,11 +831,13 @@ targets:
options: {ignore_logo: false, share_live_streams: true, remove_duplicates: false}
```
- The Web UI now includes a login feature for playlist users, allowing them to set their groups for filtering and managing their own bouquet of groups.
- The Web UI now includes a login feature for playlist users, allowing them to set their groups for filtering and managing their own bouquet of
groups.
The playlist user can login with his credentials and can select the desired groups for his playlist.
- Added `user_config_dir` to `config.yml`. It is the storage path for user configurations (f.e. bouquets).
- New Filter field `input` can be used along `name`, `group`, `title`, `url` and `type`. Input is a `regexp` filter. `input ~ "provider\-\d+"`
- New option `use_user_db` in `api-proxy.yml`. The Playlist Users are stored inside the config file `api-proxy.yml`. When you set this option to `true`
- New option `use_user_db` in `api-proxy.yml`. The Playlist Users are stored inside the config file `api-proxy.yml`. When you set this option to
`true`
the user are stored in a db file. This is a better choice if you have a lot of users. If you have only a few let it default to `false`
- WebUI playlist browser with tree and gallery mode. Explore self hosted and provider playlists in browser.
- Added HdHomeRun tuner target for use with Plex/Emby/Jellyfin
@@ -805,7 +852,8 @@ targets:
## 2.2.0 (2025-02-11)
- !BREAKING CHANGE! unique `input` `name` is now mandatory, because rearranging the `source.yml` could lead to wrong results without a playlist update.
- !BREAKING CHANGE! unique `input` `name` is now mandatory, because rearranging the `source.yml` could lead to wrong results without a playlist
update.
- !BREAKING_CHANGE! `log_sanitize_sensitive_info` is now under `log` section as `sanitize_sensitive_info`
- !BREAKING_CHANGE! uuid generation for entries changed to `input.name` + `stream_id`. Virtual id mapping changed. The new Virtual id is not a
sequence anymore.
@@ -834,18 +882,22 @@ server:
- Added `log_level` to `log` config. Priority: CLI-Argument, Env-Var, Config, Default(`info`)
```yaml
log:
sanitize_sensitive_info: false
active_clients: true
log_level: debug
update_on_boot: false
web_ui_enabled: true
```
- Added new option to `input` `xtream_live_stream_without_extension`. Default is `false`. Some providers don't like `.ts` extension,
some providers need it. Now you can disable or enable it for a provider.
- Aded new option to `input` `xtream_live_stream_use_prefix`.. Default is `true`. Some providers don't like `/live/` prefix for streams,
some providers need it. Now you can disable or enable it for a provider.
- Added new option to `input` `xtream_live_stream_without_extension`. Default is `false`. Some providers don't like `.ts` extension, some providers
need it.
Now you can disable or enable it for a provider.
- Aded new option to `input` `xtream_live_stream_use_prefix`.. Default is `true`. Some providers don't like `/live/` prefix for streams, some
providers need it.
Now you can disable or enable it for a provider.
- Added `path` to `api-proxy.yml` server config for simpler front reverse-proxy configuration (like nginx)
- added `hlsr` handling.
- fixed mapper counter not incrementing.
@@ -859,9 +911,11 @@ web_ui_enabled: true
- Added option `channel_unavailable_file` in `config.yml`. If a provider stream is not available this file content is send instead.
```yaml
update_on_boot: false
web_ui_enabled: true
channel_unavailable_file: /freeze_frame.ts
```
## 2.1.3 (2025-01-26)
@@ -879,11 +933,13 @@ channel_unavailable_file: /freeze_frame.ts
- Added `pushover.net` config to messaging.
```yaml
messaging:
pushover:
token: _required_
user: _required_
url: `optional`, default is https://api.pushover.net/1/messages.json
```
## 2.1.1 (2025-01-19)
@@ -897,7 +953,8 @@ messaging:
## 2.1.0 (2025-01-17)
- Watch files are now moved inside the `target` folder. Move them manually from `watch_<target_name>_<watched_group>.bin` to `<target_name>/watch_<watched_group>.bin`
- Watch files are now moved inside the `target` folder. Move them manually from `watch_<target_name>_<watched_group>.bin` to
`<target_name>/watch_<watched_group>.bin`
- No error log for xtream api when content is skipped with options `xtream_skip_[live|vod|series]`
- _experimental_: added live channel connection sharing in reverse proxy mode. To activate set `share_live_streams` in target options.
- Added `info` and `tmdb-id` caching for vod and series with options `xtream_resolve_(series|vod)`.
@@ -928,6 +985,7 @@ schedules:
- `config.yml` `backup_dir` is now default `backup`. If you want to keep the old name set `backup_dir: .backup`
```yaml
reverse_proxy:
stream:
retry: true
@@ -939,11 +997,13 @@ reverse_proxy:
size: 500MB
enabled: true
dir: ./cache
```
## 2.0.10 (2024-12-03)
- added Target Output Option `m3u_include_type_in_url`, default false. This adds `live`, `movie`, `series` to the url of the stream in reverse proxy mode.
- added Target Output Option `m3u_include_type_in_url`, default false. This adds `live`, `movie`, `series` to the url of the stream in reverse proxy
mode.
- added Target Output Option `m3u_mask_redirect_url`, default false. The urls are pointed to tuliprox in redirect mode. In stream request a redirect
response is send. Usefully if you want to track calls in redirect mode.
- fixed xtream api redirect url problem.
@@ -961,7 +1021,8 @@ reverse_proxy:
## 2.0.7 (2024-11-05)
- EPG is now first downloaded to disk instead of directly into memory, then processed using a SAX parser (slower but reduces memory usage from up to 2GB).
- EPG is now first downloaded to disk instead of directly into memory, then processed using a SAX parser (slower but reduces memory usage from up to
2GB).
- Various code optimizations have been applied.
- Regular expression matching in log output is now set to trace level to prevent flooding the debug log.
- Processing stats now include a `took` field indicating the processing time.
@@ -973,12 +1034,15 @@ reverse_proxy:
- api responses are now streamed directly from disk to avoid memory allocation.
- fixed scheduler implementation to only wake up on scheduled times.
-
## 2.0.5(2024-10-16)
- input url supports now scheme `file://...` (which is not necessary because file paths are supported). Gzip files are also supported.
- sort takes now a sequence for channel values which has higher priority than sort order
- fixed error handling in filter parsing
- `NOT` filter is now `non greedy`. `NOT Name ~ "A" AND Group ~ "B"` was `NOT (Name ~ "A" AND Group ~ "B")`. Now it is `(NOT Name ~ "A") AND Group ~ "B"`
- `NOT` filter is now `non greedy`. `NOT Name ~ "A" AND Group ~ "B"` was `NOT (Name ~ "A" AND Group ~ "B")`. Now it is `(NOT Name ~ "A") AND Group ~
"B"`
- Implemented workaround for missing tvg-ID
## 2.0.4(2024-09-19)
@@ -999,18 +1063,18 @@ reverse_proxy:
- added `chno` to Playlist attributes.
- `epg_channel_id` mapping fixed
## 2.0.2(2024-05-28)
## v2.0.2(2024-05-28)
- Added Encoding handling: gzip,deflate
- Fixed panic when `tvg-id` is not set.
## 2.0.1(2024-05-24)
## v2.0.1(2024-05-24)
- m3u playlists are not saved as plainfile, therefor m3u output filename is not mandatory, if given the plain m3u playlist is stored.
- Added `--healthcheck` argument for docker
- Added `catch-up`/`timeshift` api for `xtream`
## 2.0.0(2024-05-10)
## v2.0.0(2024-05-10)
- major version change due to massive changes
- `update_on_boot` for config, default is false, if true an update is run on start
@@ -1027,7 +1091,8 @@ reverse_proxy:
- Generating xtream stream urls from m3u input.
- Reverse proxy implementation for m3u playlist.
- Mapper can now set `epg_channel_id`.
- Added environment variables for User Credentials `username`, `password` and `token` in format `${env:<EnvVarName>}` where `<EnvVarName>` should be replaced.
- Added environment variables for User Credentials `username`, `password` and `token` in format `${env:<EnvVarName>}` where `<EnvVarName>` should be
replaced.
- Added `web_ui_enabled` to `config.yml`. Default is `true`. Set to `false` to disable webui.
- Added `web_auth` to `config.yml` struct for web-ui-authentication is optional.
- `enabled`: default true
@@ -1084,7 +1149,7 @@ reverse_proxy:
- `organize_into_directories`
- `episode_pattern`
- Web-UI - Download View with multi download support
- Added WebSearch Url `web_search: 'https://www.imdb.com/search/title/?title={}'` under video configuration.
- Added WebSearch Url `web_search: '<\1> under video configuration.
## v1.1.2(2023-11-03)
@@ -1142,6 +1207,7 @@ user:
- target: pl1
credentials:
- {username: x3452, password: ztrhgrGZrt83hjerter}
```
## v1.0.1(2023-09-07)
@@ -1228,11 +1294,13 @@ user:
- MRF
default is FMR
- Added mapping parameter `match_as_ascii`. Default is `false`.
If `true` before regexp matching the matching text will be converted to ascii. [unidecode](https://chowdhurya.github.io/rust-unidecode/unidecode/index.html)
If `true` before regexp matching the matching text will be converted to ascii.
[unidecode](https://chowdhurya.github.io/rust-unidecode/unidecode/index.html)
Added regexp templates to mapper:
```yaml
mappings:
- id: France
tag: ""
@@ -1257,7 +1325,7 @@ mappings:
- `mapping` attribute for target is now a list. You can assign multiple mapper to a target.
```yaml
```text
mapping:
- France
- Belgium
@@ -1268,6 +1336,7 @@ mapping:
- Added mappings. Mappings are defined in a file named ```mapping.yml``` or can be given by command line option ```-m```.
```target``` has now an optional field ```mapping``` which has the id of the mapping configuration.
- rename is now optional
## v0.9.3(2022-04-21)
@@ -1282,8 +1351,8 @@ mapping:
## v0.9.1(2022-04-05)
- There are two types of targets ```m3u``` and ```strm```. This can be set by the ```output``` attribute to ```Strm``` or ```M3u```.
If the attribute is not specified ```M3u``` is created by default. ```Strm``` output has an additional option ```underscore_whitespace```.
This replaces all whitespaces with ```_``` in the path.
If the attribute is not specified ```M3u``` is created by default. ```Strm``` output has an additional option
```underscore_whitespace```. This replaces all whitespaces with ```_``` in the path.
## v0.9.0(2022-04-04)
@@ -1325,7 +1394,7 @@ mapping:
## v0.2.0(2021-10-07)
- Added simple WEB-UI
- Start in server mode
- Start in server mode
## v0.1.0(2021-10-01)
Cargo.lock
Generated
+351 -195
View File
File diff suppressed because it is too large Load Diff
README.md
+190 -93
View File
File diff suppressed because it is too large Load Diff
backend/Cargo.toml
+3 -3
View File
@@ -24,7 +24,7 @@ axum = { version = "0.8" , features = ["macros", "default", "ws"]}
tower = "0.5"
tower-http = { version = "0.6", features = ["cors", "auth", "fs", "compression-full", "trace"] }
tower_governor = { version = "0.8", features = ["axum"] }
jsonwebtoken = { version = "10.2", features = ["rust_crypto"] }
jsonwebtoken = { version = "10.3", features = ["rust_crypto"] }
rust-argon2 = "3"
futures = "0.3"
path-clean = "1.0"
@@ -50,7 +50,7 @@ tokio = { version = "1.49", features = ["rt-multi-thread", "parking_lot", "fs",
#tracing = "0.1"
#tracing-subscriber = { version = "0.3", features = ["fmt", "env-filter"] }
tokio-util = { version = "0.7", features = ["io", "io-util"] }
tempfile = "3.24"
tempfile = "3.25"
filetime = "0.2"
strsim = "0.11"
rphonetic = "3.0"
@@ -89,7 +89,7 @@ windows-sys = { version = "0.59", features = ["Win32_Foundation", "Win32_System_
# tikv-jemallocator = { version = "*", features = ["profiling", "unprefixed_malloc_on_supported_platforms"] }
[build-dependencies]
vergen = { version = "9.0", features = ["build"] }
vergen = { version = "9.1", features = ["build"] }
[dev-dependencies]
http-body-util = "0.1"
backend/src/api/api_utils.rs
+295 -394
View File
File diff suppressed because it is too large Load Diff
backend/src/api/endpoints/hdhomerun_api.rs
+7 -7
View File
@@ -1,7 +1,7 @@
use crate::api::api_utils::{try_unwrap_body, internal_server_error};
use crate::api::model::HdHomerunAppState;
use crate::auth::AuthBasic;
use crate::model::{AppConfig, ConfigTarget, ProxyUserCredentials};
use crate::model::{AppConfig, ConfigInputFlags, ConfigTarget, ProxyUserCredentials};
use crate::utils::arc_str_serde;
use crate::processing::parser::xtream::get_xtream_url;
use crate::repository::{iter_raw_m3u_target_playlist, M3uPlaylistIterator};
@@ -103,13 +103,13 @@ where
match channels {
Some(chans) => {
let mapped = chans.map(move |(item, has_next)| {
let input_options = cfg.get_input_options_by_name(&item.input_name);
let (live_stream_use_prefix, live_stream_without_extension) = input_options
let input = cfg.get_input_by_name(&item.input_name);
let (live_stream_use_prefix, live_stream_without_extension) = input
.as_ref()
.map_or((true, false), |o| {
.map_or((true, false), |i| {
(
o.xtream_live_stream_use_prefix,
o.xtream_live_stream_without_extension,
i.has_flag(ConfigInputFlags::XtreamLiveStreamUsePrefix),
i.has_flag(ConfigInputFlags::XtreamLiveStreamWithoutExtension),
)
});
let container_extension = item.get_container_extension();
@@ -498,4 +498,4 @@ pub fn hdhr_api_register(basic_auth: bool) -> axum::Router<Arc<HdHomerunAppState
"/tuner{tuner_num}/{channel}",
axum::routing::get(auto_channel),
)
}
}
backend/src/api/endpoints/v1_api_playlist.rs
+13 -15
View File
@@ -3,7 +3,7 @@ use crate::api::endpoints::api_playlist_utils::{get_playlist_for_custom_provider
use crate::api::endpoints::extract_accept_header::ExtractAcceptHeader;
use crate::api::model::AppState;
use crate::auth::create_access_token;
use crate::model::{parse_xmltv_for_web_ui_from_url, ConfigInput, ConfigInputOptions};
use crate::model::{parse_xmltv_for_web_ui_from_url, ConfigInput, ConfigInputFlags, ConfigInputFlagsSet, ConfigInputOptions};
use axum::response::IntoResponse;
use axum::{Router};
use log::{debug, error};
@@ -25,13 +25,12 @@ fn create_config_input_for_m3u(url: &str) -> ConfigInput {
url: String::from(url),
enabled: true,
options: Some(ConfigInputOptions {
xtream_skip_live: false,
xtream_skip_vod: false,
xtream_skip_series: false,
xtream_live_stream_without_extension: false,
xtream_live_stream_use_prefix: true,
resolve_tmdb: false,
probe_stream: false,
flags: ConfigInputFlagsSet::from_variants(&[
ConfigInputFlags::XtreamLiveStreamUsePrefix,
ConfigInputFlags::ResolveBackground,
]),
resolve_delay: 2,
probe_live_interval_hours: 120,
}),
..Default::default()
}
@@ -47,13 +46,12 @@ fn create_config_input_for_xtream(username: &str, password: &str, host: &str) ->
password: Some(String::from(password)),
enabled: true,
options: Some(ConfigInputOptions {
xtream_skip_live: false,
xtream_skip_vod: false,
xtream_skip_series: false,
xtream_live_stream_without_extension: false,
xtream_live_stream_use_prefix: true,
resolve_tmdb: false,
probe_stream: false,
flags: ConfigInputFlagsSet::from_variants(&[
ConfigInputFlags::XtreamLiveStreamUsePrefix,
ConfigInputFlags::ResolveBackground,
]),
resolve_delay: 2,
probe_live_interval_hours: 120,
}),
..Default::default()
}
backend/src/api/endpoints/xtream_api.rs
+10 -15
View File
@@ -10,7 +10,7 @@ use crate::api::model::XtreamAuthorizationResponse;
use crate::api::model::{create_custom_video_stream_response, CustomVideoStreamType};
use crate::auth::Fingerprint;
use crate::model::{xtream_mapping_option_from_target_options, ConfigTarget};
use crate::model::{Config, ConfigInput};
use crate::model::{Config, ConfigInput, ConfigInputFlags};
use crate::model::{InputSource, ProxyUserCredentials};
use crate::repository::get_target_storage_path;
use crate::repository::storage_const;
@@ -153,10 +153,7 @@ pub(in crate::api) fn get_xtream_player_api_stream_url(
if let Some(input_user_info) = input.get_user_info() {
let ctx = match context {
ApiStreamContext::LiveAlt | ApiStreamContext::Live => {
let use_prefix = input
.options
.as_ref()
.is_none_or(|o| o.xtream_live_stream_use_prefix);
let use_prefix = input.has_flag(ConfigInputFlags::XtreamLiveStreamUsePrefix);
String::from(if use_prefix { "live" } else { "" })
}
ApiStreamContext::Movie | ApiStreamContext::Series | ApiStreamContext::Timeshift => {
@@ -404,8 +401,8 @@ async fn xtream_player_api_stream(
fn get_query_path(action_path: &str, stream_ext: Option<&String>, pli: &XtreamPlaylistItem, app_state: &Arc<AppState>) -> (String, String) {
let discard_extension = if pli.item_type.is_live() {
app_state.app_config.sources.load().get_input_by_name(&pli.input_name)
.as_ref().and_then(|i| i.options.as_ref())
.is_some_and(|o| o.xtream_live_stream_without_extension)
.as_ref()
.is_some_and(|i| i.has_flag(ConfigInputFlags::XtreamLiveStreamWithoutExtension))
} else {
false
};
@@ -1185,14 +1182,12 @@ async fn xtream_player_api(
let (skip_live, skip_vod, skip_series) =
if let Some(inputs) = app_state.app_config.get_inputs_for_target(&target.name) {
inputs.iter().fold((true, true, true), |acc, i| {
let (l, v, s) = acc;
i.options.as_ref().map_or((false, false, false), |o| {
(
l && o.xtream_skip_live,
v && o.xtream_skip_vod,
s && o.xtream_skip_series,
)
})
let (live, vod, series) = acc;
(
live && i.has_flag(ConfigInputFlags::XtreamSkipLive),
vod && i.has_flag(ConfigInputFlags::XtreamSkipVod),
series && i.has_flag(ConfigInputFlags::XtreamSkipSeries),
)
})
} else {
(false, false, false)
backend/src/api/hdhomerun_proprietary.rs
+3 -3
View File
@@ -1,5 +1,5 @@
use crate::api::model::AppState;
use crate::model::{AppConfig, HdHomeRunDeviceConfig};
use crate::model::{AppConfig, HdHomeRunDeviceConfig, HdHomeRunFlags};
use bytes::{Buf, BufMut, BytesMut};
use log::{error, info, trace};
use std::collections::HashMap;
@@ -236,7 +236,7 @@ async fn proprietary_discover_loop(
trace!("Received proprietary HDHomeRun discover from {remote_addr}");
let hdhomerun_guard = app_config.hdhomerun.load();
if let Some(hd_config) = &*hdhomerun_guard {
if hd_config.enabled {
if hd_config.flags.contains(HdHomeRunFlags::Enabled) {
for device in &hd_config.devices {
if device.t_enabled {
let dev_id = u32::from_str_radix(&device.device_id, 16).unwrap_or(0);
@@ -473,4 +473,4 @@ pub fn spawn_proprietary_tasks(
// TCP Get/Set Task
tokio::spawn(proprietary_tcp_listener_loop(app_state, cancel_token));
}
}
backend/src/api/hdhomerun_ssdp.rs
+3 -3
View File
@@ -1,4 +1,4 @@
use crate::model::{AppConfig, HdHomeRunDeviceConfig};
use crate::model::{AppConfig, HdHomeRunDeviceConfig, HdHomeRunFlags};
use socket2::{Domain, Protocol, Socket, Type};
use std::net::{Ipv4Addr, SocketAddr, UdpSocket as StdUdpSocket};
use std::sync::Arc;
@@ -61,7 +61,7 @@ async fn ssdp_task_loop(socket: UdpSocket, app_config: Arc<AppConfig>, server_ho
trace!("Received HDHomeRun M-SEARCH from {remote_addr}");
let hdhomerun_guard = app_config.hdhomerun.load();
if let Some(hd_config) = &*hdhomerun_guard {
if hd_config.enabled {
if hd_config.flags.contains(HdHomeRunFlags::Enabled) {
for device in &hd_config.devices {
if device.t_enabled {
let response = create_ssdp_response(device, &server_host);
@@ -116,4 +116,4 @@ pub fn spawn_ssdp_discover_task(app_config: Arc<AppConfig>, server_host: String,
Err(e) => error!("Failed to create tokio UdpSocket for SSDP discovery: {e}"),
}
});
}
}
backend/src/api/main_api.rs
+9 -6
View File
@@ -16,7 +16,7 @@ use crate::api::panel_api::sync_panel_api_exp_dates_on_boot;
use crate::api::scheduler::{exec_interner_prune, exec_scheduler};
use crate::api::serve::serve;
use crate::api::sys_usage::exec_system_usage;
use crate::model::{AppConfig, Config, Healthcheck, ProcessTargets, RateLimitConfig};
use crate::model::{AppConfig, Config, HdHomeRunFlags, Healthcheck, ProcessTargets, RateLimitConfig};
use crate::processing::processor::exec_processing;
use crate::repository::get_geoip_path;
use crate::repository::load_playlists_into_memory_cache;
@@ -182,8 +182,8 @@ pub(in crate::api) fn start_hdhomerun(
let host = config.api.host.clone();
let guard = app_config.hdhomerun.load();
if let Some(hdhomerun) = &*guard {
if hdhomerun.enabled {
if hdhomerun.ssdp_discovery {
if hdhomerun.flags.contains(HdHomeRunFlags::Enabled) {
if hdhomerun.flags.contains(HdHomeRunFlags::SsdpDiscovery) {
info!("HDHomeRun SSDP discovery is enabled.");
spawn_ssdp_discover_task(
Arc::clone(app_config),
@@ -194,7 +194,10 @@ pub(in crate::api) fn start_hdhomerun(
info!("HDHomeRun SSDP discovery is disabled.");
}
if hdhomerun.proprietary_discovery {
if hdhomerun
.flags
.contains(HdHomeRunFlags::ProprietaryDiscovery)
{
info!("HDHomeRun proprietary discovery is enabled.");
spawn_proprietary_tasks(
Arc::clone(app_state),
@@ -211,7 +214,7 @@ pub(in crate::api) fn start_hdhomerun(
let app_host = host.clone();
let port = device.port;
let device_clone = Arc::new(device.clone());
let basic_auth = hdhomerun.auth;
let basic_auth = hdhomerun.flags.contains(HdHomeRunFlags::Auth);
infos.push(format!(
"HdHomeRun Server '{}' running: http://{host}:{port}",
device.name
@@ -527,4 +530,4 @@ fn exec_input_update_listener(_app_state: &Arc<AppState>, _targets: &Arc<Process
// }
// }
// });
}
}
backend/src/api/model/app_state.rs
+35 -33
View File
@@ -12,6 +12,7 @@ use crate::utils::GeoIp;
use arc_swap::{ArcSwap, ArcSwapOption};
use log::{error, info};
use reqwest::Client;
use shared::create_bitset;
use shared::error::TuliproxError;
use shared::info_err_res;
use shared::model::UserConnectionPermission;
@@ -26,8 +27,8 @@ use tokio_util::sync::CancellationToken;
use crate::api::model::metadata_update_manager::MetadataUpdateManager;
macro_rules! cancel_service {
($field: ident, $changes:expr, $cancel_tokens:expr) => {
if $changes.$field {
($field: ident, $flag:expr, $changes:expr, $cancel_tokens:expr) => {
if $changes.flags.contains($flag) {
$cancel_tokens.$field.cancel();
CancellationToken::default()
} else {
@@ -58,18 +59,22 @@ struct TargetChanges {
target: Arc<ConfigTarget>,
}
#[allow(clippy::struct_excessive_bools)]
create_bitset!(u8, UpdateChangesFlags, Scheduler, Hdhomerun, FileWatch, Geoip);
pub(in crate::api) struct UpdateChanges {
scheduler: bool,
hdhomerun: bool,
file_watch: bool,
geoip: bool,
flags: UpdateChangesFlagsSet,
targets: Option<HashMap<String, TargetChanges>>,
}
impl UpdateChanges {
pub(in crate::api) fn modified(&self) -> bool {
self.scheduler || self.hdhomerun || self.file_watch || self.geoip
!self.flags.is_empty()
}
fn set_flag_if(&mut self, condition: bool, flag: UpdateChangesFlags) {
if condition {
self.flags.set(flag);
}
}
}
@@ -142,9 +147,9 @@ fn cancel_services(app_state: &Arc<AppState>, changes: &UpdateChanges) {
}
let cancel_tokens = app_state.cancel_tokens.load();
let scheduler = cancel_service!(scheduler, changes, cancel_tokens);
let hdhomerun = cancel_service!(hdhomerun, changes, cancel_tokens);
let file_watch = cancel_service!(file_watch, changes, cancel_tokens);
let scheduler = cancel_service!(scheduler, UpdateChangesFlags::Scheduler, changes, cancel_tokens);
let hdhomerun = cancel_service!(hdhomerun, UpdateChangesFlags::Hdhomerun, changes, cancel_tokens);
let file_watch = cancel_service!(file_watch, UpdateChangesFlags::FileWatch, changes, cancel_tokens);
let tokens = CancelTokens {
scheduler,
@@ -159,7 +164,7 @@ fn start_services(app_state: &Arc<AppState>, changes: &UpdateChanges) {
if !changes.modified() {
return;
}
if changes.scheduler {
if changes.flags.contains(UpdateChangesFlags::Scheduler) {
exec_scheduler(
&Arc::clone(&app_state.http_client.load()),
app_state,
@@ -168,7 +173,7 @@ fn start_services(app_state: &Arc<AppState>, changes: &UpdateChanges) {
);
}
if changes.hdhomerun && app_state.app_config.api_proxy.load().is_some() {
if changes.flags.contains(UpdateChangesFlags::Hdhomerun) && app_state.app_config.api_proxy.load().is_some() {
let mut infos = Vec::new();
crate::api::main_api::start_hdhomerun(
&app_state.app_config,
@@ -178,7 +183,7 @@ fn start_services(app_state: &Arc<AppState>, changes: &UpdateChanges) {
);
}
if changes.file_watch {
if changes.flags.contains(UpdateChangesFlags::FileWatch) {
exec_config_watch(app_state, &app_state.cancel_tokens.load().file_watch);
}
}
@@ -347,7 +352,7 @@ impl AppState {
self.active_provider.update_config(&self.app_config).await;
self.update_config().await?;
if changes.geoip {
if changes.flags.contains(UpdateChangesFlags::Geoip) {
let new_geoip = if use_geoip {
let path = get_geoip_path(&working_dir);
let _file_lock = self.app_config.file_locks.read_lock(&path).await;
@@ -429,13 +434,15 @@ impl AppState {
let geoip_enabled = config.is_geoip_enabled();
let geoip_enabled_old = old_config.is_geoip_enabled();
UpdateChanges {
scheduler: changed_schedules,
hdhomerun: changed_hdhomerun,
file_watch: changed_file_watch,
let mut changes = UpdateChanges {
flags: UpdateChangesFlagsSet::new(),
targets: None,
geoip: geoip_enabled != geoip_enabled_old,
}
};
changes.set_flag_if(changed_schedules, UpdateChangesFlags::Scheduler);
changes.set_flag_if(changed_hdhomerun, UpdateChangesFlags::Hdhomerun);
changes.set_flag_if(changed_file_watch, UpdateChangesFlags::FileWatch);
changes.set_flag_if(geoip_enabled != geoip_enabled_old, UpdateChangesFlags::Geoip);
changes
}
fn detect_changes_for_sources(&self, sources: &SourcesConfig) -> UpdateChanges {
@@ -498,13 +505,12 @@ impl AppState {
(file_watch_changed, target_changes)
};
UpdateChanges {
scheduler: false,
hdhomerun: false,
file_watch: file_watch_changed,
geoip: false,
let mut changes = UpdateChanges {
flags: UpdateChangesFlagsSet::new(),
targets: Some(target_changes),
}
};
changes.set_flag_if(file_watch_changed, UpdateChangesFlags::FileWatch);
changes
}
pub async fn cache_playlist(&self, target_name: &str, playlist: PlaylistStorage) {
@@ -560,11 +566,7 @@ fn schedules_changed(a: &[ScheduleConfig], b: &[ScheduleConfig]) -> bool {
}
fn hdhomerun_changed(a: &HdHomeRunConfig, b: &HdHomeRunConfig) -> bool {
if a.enabled != b.enabled
|| a.auth != b.auth
|| a.ssdp_discovery != b.ssdp_discovery
|| a.proprietary_discovery != b.proprietary_discovery
{
if a.flags != b.flags {
return true;
}
if !small_vecs_equal_unordered(a.devices.as_ref(), b.devices.as_ref()) {
@@ -582,4 +584,4 @@ pub struct HdHomerunAppState {
pub app_state: Arc<AppState>,
pub device: Arc<HdHomeRunDeviceConfig>,
pub hd_scan_state: Arc<AtomicI8>,
}
}
backend/src/api/model/metadata_update_manager.rs
+6 -6
View File
@@ -4,8 +4,8 @@ use crate::processing::processor::{update_generic_stream_metadata, update_live_s
use crate::utils::debug_if_enabled;
use dashmap::mapref::entry::Entry;
use dashmap::DashMap;
use log::{debug, error, info, trace, warn};
use shared::create_bit_set;
use log::{debug, error, info, warn};
use shared::create_bitset;
use shared::error::TuliproxError;
use shared::model::{InputType, LiveStreamProperties, PlaylistItemType, SeriesStreamProperties, UUIDType, VideoStreamProperties, XtreamCluster, XtreamPlaylistItem};
use shared::utils::generate_playlist_uuid;
@@ -21,7 +21,6 @@ use crate::repository::{persist_input_live_info_batch, persist_input_series_info
use crate::utils::FileReadGuard;
use std::cmp::min;
use std::collections::{HashMap, HashSet};
use crate::utils::trace_if_enabled;
const QUEUE_LOG_INTERVAL: Duration = Duration::from_secs(30);
const PROGRESS_LOG_INTERVAL: Duration = Duration::from_secs(15);
@@ -31,7 +30,7 @@ const MAX_QUEUE_SIZE: usize = 100_000;
const TASK_ERR_NO_CONNECTION: &str = "No connection available";
const TASK_ERR_PREEMPTED: &str = "Task preempted";
create_bit_set!(u32, ResolveReason, Info, Tmdb, Date, Probe, MissingDetails);
create_bitset!(u8, ResolveReason, Info, Tmdb, Date, Probe, MissingDetails);
/// `PlaylistItemIdType` ID can be either a String (M3U) or u32 (Xtream/TargetDB)
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
@@ -398,7 +397,7 @@ impl InputWorker {
} else if Self::is_series_task_key(&current_key) {
processed_series_count += 1;
}
trace!("Processed metadata task for input {input_name}: {current_task} (changed={task_changed})");
debug!("Processed metadata task for input {input_name}: {current_task} (changed={task_changed})");
retry_attempts.remove(&current_key);
if last_progress_log_at.elapsed() >= PROGRESS_LOG_INTERVAL {
// current_key is removed from pending_tasks later in this loop iteration;
@@ -1203,7 +1202,7 @@ impl InputWorker {
.and_then(|handle| handle.allocation.get_provider_config());
let name_display = item_title.as_deref().map_or(String::new(), |n| format!(" \"{n}\""));
trace_if_enabled!("Processing task for {}: {}{}", input_name, task, name_display);
debug!("Processing task for {input_name}: {task}{name_display}");
let pre_vod_updates = collector.vod.len();
let pre_series_updates = collector.series.len();
@@ -1308,6 +1307,7 @@ impl InputWorker {
item_title,
false, // Batch collect
fetch_info,
reason.contains(ResolveReason::Probe),
query_opt,
).await {
Ok(Some(props)) => {
backend/src/api/model/streams/provider_stream_factory.rs
+37 -18
View File
@@ -12,6 +12,7 @@ use futures::{StreamExt, TryStreamExt};
use log::{debug, log_enabled, warn};
use reqwest::header::{HeaderMap, RANGE};
use reqwest::StatusCode;
use shared::create_bitset;
use shared::model::{PlaylistItemType, DEFAULT_USER_AGENT};
use shared::utils::{filter_request_header, sanitize_sensitive_info};
use std::collections::HashMap;
@@ -26,21 +27,26 @@ use crate::api::model::streams::client_stream::ClientStream;
const RETRY_SECONDS: u64 = 5;
const ERR_MAX_RETRY_COUNT: u32 = 5;
#[allow(clippy::struct_excessive_bools)]
create_bitset!(
u8,
ProviderStreamFactoryFlags,
ReconnectEnabled,
BufferEnabled,
ShareStream,
PipeStream,
RangeRequested
);
#[derive(Debug, Clone)]
pub struct ProviderStreamFactoryOptions {
addr: SocketAddr,
// item_type: PlaylistItemType,
reconnect_enabled: bool,
buffer_enabled: bool,
flags: ProviderStreamFactoryFlagsSet,
buffer_size: usize,
share_stream: bool,
pipe_stream: bool,
url: Url,
headers: HeaderMap,
default_user_agent: Option<axum::http::header::HeaderValue>,
range_bytes: Arc<Option<AtomicUsize>>,
range_requested: bool,
reconnect_flag: Arc<AtomicOnceFlag>,
provider: Option<Arc<ConfigProvider>>,
}
@@ -89,21 +95,33 @@ impl ProviderStreamFactoryOptions {
} else {
Arc::new(Some(AtomicUsize::new(requested_range.unwrap_or(0))))
};
let mut flags = ProviderStreamFactoryFlagsSet::new();
if stream_options.stream_retry {
flags.set(ProviderStreamFactoryFlags::ReconnectEnabled);
}
if stream_options.pipe_provider_stream {
flags.set(ProviderStreamFactoryFlags::PipeStream);
}
if stream_options.buffer_enabled {
flags.set(ProviderStreamFactoryFlags::BufferEnabled);
}
if share_stream {
flags.set(ProviderStreamFactoryFlags::ShareStream);
}
if requested_range.is_some() {
flags.set(ProviderStreamFactoryFlags::RangeRequested);
}
Self {
// item_type,
addr,
reconnect_enabled: stream_options.stream_retry,
pipe_stream: stream_options.pipe_provider_stream,
buffer_enabled: stream_options.buffer_enabled,
flags,
buffer_size,
share_stream,
reconnect_flag: Arc::new(AtomicOnceFlag::new()),
url,
headers,
default_user_agent,
range_bytes,
range_requested: requested_range.is_some(),
provider: None,
}
}
@@ -118,17 +136,17 @@ impl ProviderStreamFactoryOptions {
#[inline]
fn is_piped(&self) -> bool {
self.pipe_stream
self.flags.contains(ProviderStreamFactoryFlags::PipeStream)
}
#[inline]
fn is_buffer_enabled(&self) -> bool {
self.buffer_enabled
self.flags.contains(ProviderStreamFactoryFlags::BufferEnabled)
}
#[inline]
fn is_shared_stream(&self) -> bool {
self.share_stream
self.flags.contains(ProviderStreamFactoryFlags::ShareStream)
}
#[inline]
@@ -158,7 +176,8 @@ impl ProviderStreamFactoryOptions {
#[inline]
pub fn should_reconnect(&self) -> bool {
self.reconnect_enabled
self.flags
.contains(ProviderStreamFactoryFlags::ReconnectEnabled)
}
#[inline]
@@ -190,7 +209,7 @@ impl ProviderStreamFactoryOptions {
#[inline]
pub fn was_range_requested(&self) -> bool {
self.range_requested
self.flags.contains(ProviderStreamFactoryFlags::RangeRequested)
}
}
@@ -612,9 +631,9 @@ pub async fn create_provider_stream(
Ok(Some((init_stream, info))) => {
let is_media_stream_or_not_piped = if let Some((headers, _, _, _custom_video_type)) = &info {
// if it is piped or no video stream, then we don't reconnect
!stream_options.pipe_stream && classify_content_type(headers) == MimeCategory::Video
!stream_options.is_piped() && classify_content_type(headers) == MimeCategory::Video
} else {
!stream_options.pipe_stream // don't know what it is but lets assume it is something
!stream_options.is_piped() // don't know what it is but lets assume it is something
};
let continue_signal = stream_options.get_reconnect_flag_clone();
backend/src/api/model/xtream.rs
+10 -2
View File
@@ -87,8 +87,16 @@ impl XtreamAuthorizationResponse {
},
server_info: XtreamServerInfoResponse {
url: server_info.host.clone(),
port: if server_info.protocol == "http" { server_info.port.as_ref().map_or("80", |v| v.as_str()).to_string() } else { String::from("80") },
https_port: if server_info.protocol == "https" { server_info.port.as_ref().map_or("443", |v| v.as_str()).to_string() } else { String::from("443") },
port: if server_info.protocol == "http" {
server_info.port.as_deref().unwrap_or("80").to_string()
} else {
String::from("80")
},
https_port: if server_info.protocol == "https" {
server_info.port.as_deref().unwrap_or("443").to_string()
} else {
String::from("443")
},
server_protocol: server_info.protocol.clone(),
rtmp_port: String::new(),
timezone: server_info.timezone.clone(),
backend/src/api/panel_api.rs
+42 -33
View File
@@ -20,6 +20,7 @@ use log::{error, warn};
use serde::{Deserialize, Serialize};
use serde_json::Value;
use shared::concat_string;
use shared::create_bitset;
use shared::error::{info_err, info_err_res, TuliproxError};
use shared::model::{
ConfigInputAliasDto, InputType, PanelApiAliasPoolSizeValue,
@@ -328,45 +329,53 @@ fn validate_client_adult_content_params(
Ok(())
}
#[allow(clippy::struct_excessive_bools)]
#[derive(Debug, Clone, Copy)]
struct PanelApiOptionalFlags {
account_info: bool,
client_new: bool,
client_renew: bool,
adult_content: bool,
}
create_bitset!(
u8,
PanelApiOptionalFlags,
AccountInfo,
ClientNew,
ClientRenew,
AdultContent
);
fn resolve_panel_api_optional_flags(
cfg: &PanelApiConfig,
input_name: &str,
) -> PanelApiOptionalFlags {
let flags = PanelApiOptionalFlags {
account_info: !cfg.query_parameter.account_info.is_empty(),
client_new: !cfg.query_parameter.client_new.is_empty(),
client_renew: !cfg.query_parameter.client_renew.is_empty(),
adult_content: !cfg.query_parameter.client_adult_content.is_empty(),
};
) -> PanelApiOptionalFlagsSet {
let mut flags = PanelApiOptionalFlagsSet::new();
if !cfg.query_parameter.account_info.is_empty() {
flags.set(PanelApiOptionalFlags::AccountInfo);
}
if !cfg.query_parameter.client_new.is_empty() {
flags.set(PanelApiOptionalFlags::ClientNew);
}
if !cfg.query_parameter.client_renew.is_empty() {
flags.set(PanelApiOptionalFlags::ClientRenew);
}
if !cfg.query_parameter.client_adult_content.is_empty() {
flags.set(PanelApiOptionalFlags::AdultContent);
}
let name = sanitize_sensitive_info(input_name);
if !flags.client_renew {
if !flags.contains(PanelApiOptionalFlags::ClientRenew) {
debug_if_enabled!(
"panel_api request for client_renew disabled due to missing arguments for {}",
name
);
}
if !flags.client_new {
if !flags.contains(PanelApiOptionalFlags::ClientNew) {
debug_if_enabled!(
"panel_api request for client_new disabled due to missing arguments for {}",
name
);
}
if !flags.adult_content {
if !flags.contains(PanelApiOptionalFlags::AdultContent) {
debug_if_enabled!(
"panel_api request for client_adult_content disabled due to missing arguments for {}",
name
);
}
if !flags.account_info {
if !flags.contains(PanelApiOptionalFlags::AccountInfo) {
debug_if_enabled!(
"panel_api request for account_info disabled due to missing arguments for {}",
name
@@ -1798,12 +1807,12 @@ async fn try_renew_expired_account(
is_batch: bool,
sources_path: &Path,
treat_missing_exp_date_as_expired: bool,
optional: PanelApiOptionalFlags,
optional: PanelApiOptionalFlagsSet,
) -> Option<PanelApiProvisionOutcome> {
if !optional.client_renew {
if !optional.contains(PanelApiOptionalFlags::ClientRenew) {
return None;
}
let adult_enabled = optional.adult_content;
let adult_enabled = optional.contains(PanelApiOptionalFlags::AdultContent);
let mut candidates = collect_accounts(input);
for acct in &mut candidates {
if treat_missing_exp_date_as_expired && acct.exp_date.is_none() {
@@ -1937,12 +1946,12 @@ async fn try_create_new_account(
panel_cfg: &PanelApiConfig,
is_batch: bool,
sources_path: &Path,
optional: PanelApiOptionalFlags,
optional: PanelApiOptionalFlagsSet,
) -> Option<PanelApiProvisionOutcome> {
if !optional.client_new {
if !optional.contains(PanelApiOptionalFlags::ClientNew) {
return None;
}
let adult_enabled = optional.adult_content;
let adult_enabled = optional.contains(PanelApiOptionalFlags::AdultContent);
match panel_client_new(app_state, panel_cfg).await {
Ok((username, password, base_url_from_resp)) => {
let base_url = base_url_from_resp.unwrap_or_else(|| input.url.clone());
@@ -2154,15 +2163,15 @@ async fn ensure_alias_pool_min(
sources_yml_patches: &mut Vec<SourcesYmlPatch>,
time_ctx: Option<&PanelApiTimeContext>,
effective_now: u64,
optional: PanelApiOptionalFlags,
optional: PanelApiOptionalFlagsSet,
) -> (bool, u16) {
if min_pool == 0 {
return (false, 0);
}
let renew_enabled = optional.client_renew;
let new_enabled = optional.client_new;
let adult_enabled = optional.adult_content;
let renew_enabled = optional.contains(PanelApiOptionalFlags::ClientRenew);
let new_enabled = optional.contains(PanelApiOptionalFlags::ClientNew);
let adult_enabled = optional.contains(PanelApiOptionalFlags::AdultContent);
if !renew_enabled && !new_enabled {
return (false, 0);
}
@@ -2620,9 +2629,9 @@ async fn sync_panel_api_for_input_on_boot(
.unwrap_or(0);
let min_pool = resolve_alias_pool_min(app_state.as_ref(), &input.name, panel_cfg);
let renew_enabled = optional.client_renew;
let new_enabled = optional.client_new;
let adult_enabled = optional.adult_content;
let renew_enabled = optional.contains(PanelApiOptionalFlags::ClientRenew);
let new_enabled = optional.contains(PanelApiOptionalFlags::ClientNew);
let adult_enabled = optional.contains(PanelApiOptionalFlags::AdultContent);
let provisioning_enabled = renew_enabled || new_enabled;
// Refresh/provision credentials on boot/update.
// Root is handled first (may affect desired_aliases and avoids over-provisioning).
@@ -3592,7 +3601,7 @@ async fn sync_panel_api_for_input_on_boot(
}
}
if optional.account_info {
if optional.contains(PanelApiOptionalFlags::AccountInfo) {
let creds = accounts
.first()
.map(|acct| (acct.username.as_str(), acct.password.as_str()));
backend/src/model/config/app.rs
+5 -3
View File
@@ -1,5 +1,5 @@
use crate::api::model::TransportStreamBuffer;
use crate::model::{ApiProxyConfig, ApiProxyServerInfo, Config, ConfigInput, ConfigInputOptions, ConfigTarget, CustomStreamResponse, GracePeriodOptions, HdHomeRunConfig, Mappings, ProxyUserCredentials, ReverseProxyDisabledHeaderConfig, SourcesConfig, TargetOutput};
use crate::model::{ApiProxyConfig, ApiProxyServerInfo, Config, ConfigInput, ConfigInputOptions, ConfigTarget, CustomStreamResponse, GracePeriodOptions, HdHomeRunConfig, HdHomeRunFlags, Mappings, ProxyUserCredentials, ReverseProxyDisabledHeaderConfig, SourcesConfig, TargetOutput};
use crate::utils;
use crate::utils::ffmpeg::check_ffprobe_availability;
use arc_swap::{ArcSwap, ArcSwapOption};
@@ -98,7 +98,10 @@ impl AppConfig {
let check_homerun = {
let config = self.config.load();
self.hdhomerun.store(config.hdhomerun.as_ref().map(|h| Arc::new(h.clone())));
config.hdhomerun.as_ref().is_some_and(|h| h.enabled)
config
.hdhomerun
.as_ref()
.is_some_and(|h| h.flags.contains(HdHomeRunFlags::Enabled))
};
let sources = self.sources.load();
for source in &sources.sources {
@@ -437,4 +440,3 @@ impl AppConfig {
}).await
}
}
backend/src/model/config/base.rs
+3 -3
View File
@@ -1,5 +1,5 @@
use crate::model::{macros, ConfigApi, LibraryConfig, ReverseProxyConfig, ReverseProxyDisabledHeaderConfig, ScheduleConfig};
use crate::model::{HdHomeRunConfig, IpCheckConfig, LogConfig, MessagingConfig, ProxyConfig, VideoConfig, WebUiConfig};
use crate::model::{HdHomeRunConfig, HdHomeRunFlags, IpCheckConfig, LogConfig, MessagingConfig, ProxyConfig, VideoConfig, WebUiConfig};
use crate::utils;
use log::{error, info};
use path_clean::PathClean;
@@ -146,7 +146,7 @@ impl Config {
pub fn get_hdhr_device_overview(&self) -> Option<HdHomeRunDeviceOverview> {
self.hdhomerun.as_ref().map(|hdhr|
HdHomeRunDeviceOverview {
enabled: hdhr.enabled,
enabled: hdhr.flags.contains(HdHomeRunFlags::Enabled),
devices: hdhr.devices.iter().map(|d| d.name.clone()).collect::<Vec<String>>(),
})
}
@@ -206,4 +206,4 @@ impl From<&ConfigDto> for Config {
library: dto.library.as_ref().map(Into::into),
}
}
}
}
backend/src/model/config/hdhomerun.rs
+21 -10
View File
@@ -1,5 +1,6 @@
use shared::model::{HdHomeRunConfigDto, HdHomeRunDeviceConfigDto};
use crate::model::macros;
use shared::{apply_flags, create_bitset};
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct HdHomeRunDeviceConfig {
@@ -41,25 +42,35 @@ impl From<&HdHomeRunDeviceConfigDto> for HdHomeRunDeviceConfig {
}
}
create_bitset!(
u8,
HdHomeRunFlags,
Enabled,
Auth,
SsdpDiscovery,
ProprietaryDiscovery
);
#[derive(Debug, Clone, PartialEq, Eq)]
#[allow(clippy::struct_excessive_bools)]
pub struct HdHomeRunConfig {
pub enabled: bool,
pub auth: bool,
pub ssdp_discovery: bool,
pub proprietary_discovery: bool,
pub flags: HdHomeRunFlagsSet,
pub devices: Vec<HdHomeRunDeviceConfig>,
}
macros::from_impl!(HdHomeRunConfig);
impl From<&HdHomeRunConfigDto> for HdHomeRunConfig {
fn from(dto: &HdHomeRunConfigDto) -> Self {
let mut flags = HdHomeRunFlagsSet::new();
apply_flags!(
dto, flags, HdHomeRunFlags;
(enabled, Enabled),
(auth, Auth),
(ssdp_discovery, SsdpDiscovery),
(proprietary_discovery, ProprietaryDiscovery)
);
Self {
enabled: dto.enabled,
auth: dto.auth,
ssdp_discovery: dto.ssdp_discovery,
proprietary_discovery: dto.proprietary_discovery,
flags,
devices: dto.devices.iter().map(Into::into).collect(),
}
}
}
}
backend/src/model/config/input.rs
+118 -15
View File
@@ -2,6 +2,7 @@ use crate::model::{macros, ConfigProvider, EpgConfig, PanelApiConfig};
use crate::repository::get_csv_file_path;
use chrono::Utc;
use log::warn;
use shared::{apply_flags, create_bitset};
use shared::error::TuliproxError;
use shared::model::{ConfigInputAliasDto, ConfigInputDto, ConfigInputOptionsDto, InputFetchMethod, InputType, StagedInputDto};
use shared::utils::{get_credentials_from_url, parse_provider_scheme_url_parts, sanitize_sensitive_info, Internable, PROVIDER_SCHEME_PREFIX};
@@ -14,29 +15,101 @@ use std::path::PathBuf;
use std::sync::Arc;
use url::Url;
#[allow(clippy::struct_excessive_bools)]
create_bitset!(
u16,
ConfigInputFlags,
XtreamSkipLive,
XtreamSkipVod,
XtreamSkipSeries,
XtreamLiveStreamUsePrefix,
XtreamLiveStreamWithoutExtension,
ResolveTmdb,
ResolveBackground,
ResolveSeries,
ResolveVod,
ProbeSeries,
ProbeVod,
ProbeLive
);
impl ConfigInputFlagsSet {
#[inline]
pub fn contains_any(&self, other: Self) -> bool {
(self.0 & other.0) != 0
}
#[inline]
pub fn contains_all(&self, other: Self) -> bool {
(self.0 & other.0) == other.0
}
}
impl std::ops::BitOr for ConfigInputFlags {
type Output = ConfigInputFlagsSet;
#[inline]
fn bitor(self, rhs: Self) -> Self::Output {
ConfigInputFlagsSet::from_variants(&[self, rhs])
}
}
impl std::ops::BitOr<ConfigInputFlags> for ConfigInputFlagsSet {
type Output = Self;
#[inline]
fn bitor(mut self, rhs: ConfigInputFlags) -> Self::Output {
self.set(rhs);
self
}
}
#[derive(Debug, Clone)]
pub struct ConfigInputOptions {
pub xtream_skip_live: bool,
pub xtream_skip_vod: bool,
pub xtream_skip_series: bool,
pub xtream_live_stream_use_prefix: bool,
pub xtream_live_stream_without_extension: bool,
pub resolve_tmdb: bool,
pub probe_stream: bool,
pub flags: ConfigInputFlagsSet,
pub resolve_delay: u16,
pub probe_live_interval_hours: u32,
}
macros::from_impl!(ConfigInputOptions);
impl ConfigInputOptions {
#[inline]
pub fn has_flag(&self, flag: ConfigInputFlags) -> bool {
self.flags.contains(flag)
}
#[inline]
pub fn has_any_flags(&self, flags: ConfigInputFlagsSet) -> bool {
self.flags.contains_any(flags)
}
#[inline]
pub fn has_all_flags(&self, flags: ConfigInputFlagsSet) -> bool {
self.flags.contains_all(flags)
}
}
impl From<&ConfigInputOptionsDto> for ConfigInputOptions {
fn from(dto: &ConfigInputOptionsDto) -> Self {
let mut flags = ConfigInputFlagsSet::new();
apply_flags!(
dto, flags, ConfigInputFlags;
(xtream_skip_live, XtreamSkipLive),
(xtream_skip_vod, XtreamSkipVod),
(xtream_skip_series, XtreamSkipSeries),
(xtream_live_stream_use_prefix, XtreamLiveStreamUsePrefix),
(xtream_live_stream_without_extension, XtreamLiveStreamWithoutExtension),
(resolve_tmdb, ResolveTmdb),
(resolve_background, ResolveBackground),
(resolve_series, ResolveSeries),
(resolve_vod, ResolveVod),
(probe_series, ProbeSeries),
(probe_vod, ProbeVod),
(probe_live, ProbeLive)
);
Self {
xtream_skip_live: dto.xtream_skip_live,
xtream_skip_vod: dto.xtream_skip_vod,
xtream_skip_series: dto.xtream_skip_series,
xtream_live_stream_use_prefix: dto.xtream_live_stream_use_prefix,
xtream_live_stream_without_extension: dto.xtream_live_stream_without_extension,
resolve_tmdb: dto.resolve_tmdb,
probe_stream: dto.probe_stream,
flags,
resolve_delay: dto.resolve_delay,
probe_live_interval_hours: dto.probe_live_interval_hours,
}
}
}
@@ -159,6 +232,36 @@ pub struct ConfigInput {
}
impl ConfigInput {
#[inline]
pub fn has_flag(&self, flag: ConfigInputFlags) -> bool {
self.options.as_ref().is_some_and(|o| o.has_flag(flag))
}
#[inline]
pub fn has_flag_or(&self, flag: ConfigInputFlags, default: bool) -> bool {
self.options.as_ref().map_or(default, |o| o.has_flag(flag))
}
#[inline]
pub fn has_any_flags(&self, flags: ConfigInputFlagsSet) -> bool {
self.options.as_ref().is_some_and(|o| o.has_any_flags(flags))
}
#[inline]
pub fn has_any_flags_or(&self, flags: ConfigInputFlagsSet, default: bool) -> bool {
self.options.as_ref().map_or(default, |o| o.has_any_flags(flags))
}
#[inline]
pub fn has_all_flags(&self, flags: ConfigInputFlagsSet) -> bool {
self.options.as_ref().is_some_and(|o| o.has_all_flags(flags))
}
#[inline]
pub fn has_all_flags_or(&self, flags: ConfigInputFlagsSet, default: bool) -> bool {
self.options.as_ref().map_or(default, |o| o.has_all_flags(flags))
}
pub fn prepare(&mut self, provider_configs: &[Arc<ConfigProvider>]) -> Result<Option<PathBuf>, TuliproxError> {
let mut used_provider_configs: Vec<Arc<ConfigProvider>> = vec![];
let batch_file_path = self.prepare_batch();
backend/src/model/config/target.rs
+23 -38
View File
@@ -9,10 +9,10 @@ use shared::model::PlaylistItemType;
use std::sync::Arc;
use shared::foundation::Filter;
use shared::foundation::ValueProvider;
use shared::create_bit_set;
use shared::{apply_flags, create_bitset};
create_bit_set!(u16, XtreamTargetFlags, SkipLiveDirectSource, SkipVideoDirectSource, SkipSeriesDirectSource,
ResolveBackground, ResolveSeries, ResolveVod, ProbeSeries, ProbeVod, ProbeLive);
create_bitset!(u8, XtreamTargetFlags, SkipLiveDirectSource, SkipVideoDirectSource, SkipSeriesDirectSource);
create_bitset!(u8, StrmTargetFlags, Flat, UnderscoreWhitespace, Cleanup, AddQualityToFilename);
#[derive(Clone, Debug)]
pub struct ProcessTargets {
@@ -32,12 +32,9 @@ impl ProcessTargets {
}
}
#[allow(clippy::struct_excessive_bools)]
#[derive(Debug, Clone)]
pub struct XtreamTargetOutput {
pub flags: XtreamTargetFlagsSet,
pub probe_live_interval_hours: u32,
pub resolve_delay: u16,
pub trakt: Option<TraktConfig>,
pub filter: Option<Filter>,
}
@@ -46,20 +43,15 @@ macros::from_impl!(XtreamTargetOutput);
impl From<&XtreamTargetOutputDto> for XtreamTargetOutput {
fn from(dto: &XtreamTargetOutputDto) -> Self {
let mut flags = XtreamTargetFlagsSet::new();
if dto.skip_live_direct_source { flags.add(XtreamTargetFlags::SkipLiveDirectSource); }
if dto.skip_video_direct_source { flags.add(XtreamTargetFlags::SkipVideoDirectSource); }
if dto.skip_series_direct_source { flags.add(XtreamTargetFlags::SkipSeriesDirectSource); }
if dto.resolve_background { flags.add(XtreamTargetFlags::ResolveBackground); }
if dto.resolve_series { flags.add(XtreamTargetFlags::ResolveSeries); }
if dto.resolve_vod { flags.add(XtreamTargetFlags::ResolveVod); }
if dto.probe_series { flags.add(XtreamTargetFlags::ProbeSeries); }
if dto.probe_vod { flags.add(XtreamTargetFlags::ProbeVod); }
if dto.probe_live { flags.add(XtreamTargetFlags::ProbeLive); }
apply_flags!(
dto, flags, XtreamTargetFlags;
(skip_live_direct_source, SkipLiveDirectSource),
(skip_video_direct_source, SkipVideoDirectSource),
(skip_series_direct_source, SkipSeriesDirectSource)
);
Self {
flags,
resolve_delay: dto.resolve_delay,
probe_live_interval_hours: dto.probe_live_interval_hours,
trakt: dto.trakt.as_ref().map(Into::into),
filter: dto.t_filter.clone(),
}
@@ -72,14 +64,6 @@ impl From<&XtreamTargetOutput> for XtreamTargetOutputDto {
skip_live_direct_source: instance.flags.contains(XtreamTargetFlags::SkipLiveDirectSource),
skip_video_direct_source: instance.flags.contains(XtreamTargetFlags::SkipVideoDirectSource),
skip_series_direct_source: instance.flags.contains(XtreamTargetFlags::SkipSeriesDirectSource),
resolve_background: instance.flags.contains(XtreamTargetFlags::ResolveBackground),
resolve_series: instance.flags.contains(XtreamTargetFlags::ResolveSeries),
resolve_vod: instance.flags.contains(XtreamTargetFlags::ResolveVod),
probe_series: instance.flags.contains(XtreamTargetFlags::ProbeSeries),
probe_vod: instance.flags.contains(XtreamTargetFlags::ProbeVod),
probe_live: instance.flags.contains(XtreamTargetFlags::ProbeLive),
resolve_delay: instance.resolve_delay,
probe_live_interval_hours: instance.probe_live_interval_hours,
trakt: instance.trakt.as_ref().map(TraktConfigDto::from),
filter: instance.filter.as_ref().map(ToString::to_string),
t_filter: instance.filter.clone(),
@@ -120,17 +104,13 @@ impl From<&M3uTargetOutput> for M3uTargetOutputDto {
#[derive(Debug, Clone)]
#[allow(clippy::struct_excessive_bools)]
pub struct StrmTargetOutput {
pub directory: String,
pub username: Option<String>,
pub style: StrmExportStyle,
pub flat: bool,
pub underscore_whitespace: bool,
pub cleanup: bool,
pub flags: StrmTargetFlagsSet,
pub strm_props: Option<Vec<String>>,
pub filter: Option<Filter>,
pub add_quality_to_filename: bool,
pub probe_probe_size_bytes: Option<u64>,
pub probe_analyze_duration: Option<u64>,
}
@@ -138,16 +118,21 @@ pub struct StrmTargetOutput {
macros::from_impl!(StrmTargetOutput);
impl From<&StrmTargetOutputDto> for StrmTargetOutput {
fn from(dto: &StrmTargetOutputDto) -> Self {
let mut flags = StrmTargetFlagsSet::new();
apply_flags!(
dto, flags, StrmTargetFlags;
(flat, Flat),
(underscore_whitespace, UnderscoreWhitespace),
(cleanup, Cleanup),
(add_quality_to_filename, AddQualityToFilename)
);
Self {
directory: dto.directory.clone(),
username: dto.username.clone(),
style: dto.style,
flat: dto.flat,
underscore_whitespace: dto.underscore_whitespace,
cleanup: dto.cleanup,
flags,
strm_props: dto.strm_props.clone(),
filter: dto.t_filter.clone(),
add_quality_to_filename: dto.add_quality_to_filename,
probe_probe_size_bytes: dto.probe_probe_size_bytes,
probe_analyze_duration: dto.probe_analyze_duration,
}
@@ -159,13 +144,13 @@ impl From<&StrmTargetOutput> for StrmTargetOutputDto {
directory: instance.directory.clone(),
username: instance.username.clone(),
style: instance.style,
flat: instance.flat,
underscore_whitespace: instance.underscore_whitespace,
cleanup: instance.cleanup,
flat: instance.flags.contains(StrmTargetFlags::Flat),
underscore_whitespace: instance.flags.contains(StrmTargetFlags::UnderscoreWhitespace),
cleanup: instance.flags.contains(StrmTargetFlags::Cleanup),
strm_props: instance.strm_props.clone(),
filter: instance.filter.as_ref().map(ToString::to_string),
t_filter: instance.filter.clone(),
add_quality_to_filename: instance.add_quality_to_filename,
add_quality_to_filename: instance.flags.contains(StrmTargetFlags::AddQualityToFilename),
probe_probe_size_bytes: instance.probe_probe_size_bytes,
probe_analyze_duration: instance.probe_analyze_duration,
}
backend/src/model/xtream.rs
+29 -6
View File
@@ -3,7 +3,10 @@ use crate::model::{AppConfig, ProxyUserCredentials};
use crate::model::{ConfigTarget, XtreamTargetOutput};
use serde::{Deserialize, Serialize};
use serde_json::Value;
use shared::model::{PlaylistItem, PlaylistItemType, PlaylistItemTypeSet, ProxyUserStatus, XtreamMappingOptions};
use shared::model::{
PlaylistItem, PlaylistItemType, PlaylistItemTypeSet, ProxyUserStatus, XtreamMappingFlags,
XtreamMappingFlagsSet, XtreamMappingOptions,
};
use shared::utils::{arc_str_serde, deserialize_number_from_string_or_zero};
use enum_iterator::all;
use crate::model::XtreamTargetFlags;
@@ -45,11 +48,31 @@ pub fn xtream_mapping_option_from_target_options(target: &ConfigTarget, target_o
}
}
let mut flags = XtreamMappingFlagsSet::new();
if target_output
.flags
.contains(XtreamTargetFlags::SkipLiveDirectSource)
{
flags.set(XtreamMappingFlags::SkipLiveDirectSource);
}
if target_output
.flags
.contains(XtreamTargetFlags::SkipVideoDirectSource)
{
flags.set(XtreamMappingFlags::SkipVideoDirectSource);
}
if target_output
.flags
.contains(XtreamTargetFlags::SkipSeriesDirectSource)
{
flags.set(XtreamMappingFlags::SkipSeriesDirectSource);
}
if cfg.is_reverse_proxy_resource_rewrite_enabled() {
flags.set(XtreamMappingFlags::RewriteResourceUrl);
}
XtreamMappingOptions {
skip_live_direct_source: target_output.flags.contains(XtreamTargetFlags::SkipLiveDirectSource),
skip_video_direct_source: target_output.flags.contains(XtreamTargetFlags::SkipVideoDirectSource),
skip_series_direct_source: target_output.flags.contains(XtreamTargetFlags::SkipSeriesDirectSource),
rewrite_resource_url: cfg.is_reverse_proxy_resource_rewrite_enabled(),
flags,
force_redirect,
reverse_item_types,
username: user.username.clone(),
@@ -84,4 +107,4 @@ pub struct PlaylistXtreamCategory {
pub id: u32,
#[serde(alias = "category_name")]
pub name: String,
}
}
backend/src/processing/parser/xtream.rs
+27 -14
View File
@@ -1,4 +1,4 @@
use crate::model::ConfigInput;
use crate::model::{ConfigInput, ConfigInputFlags};
use crate::model::XtreamCategory;
use crate::utils::request::DynReader;
use crate::utils::xtream::get_xtream_stream_url_base;
@@ -67,8 +67,10 @@ pub fn parse_xtream_series_info(parent_uuid: &UUIDType, series_info: &SeriesStre
series_release_date: Option<&Arc<str>>,
) -> Option<Vec<PlaylistItem>> {
let url = input.url.as_str();
let username = input.username.as_ref().map_or("", |v| v);
let password = input.password.as_ref().map_or("", |v| v);
let (username, password) = (
input.username.as_deref().unwrap_or(""),
input.password.as_deref().unwrap_or(""),
);
if let Some(episodes) = series_info.details.as_ref().and_then(|d| d.episodes.as_ref()) {
let result: Vec<PlaylistItem> = episodes.iter().map(|episode| {
@@ -149,8 +151,10 @@ pub async fn parse_xtream(input: &ConfigInput,
Ok(xtream_categories) => {
let input_name = input.name.clone();
let url = input.url.as_str();
let username = input.username.as_ref().map_or("", |v| v);
let password = input.password.as_ref().map_or("", |v| v);
let (username, password) = (
input.username.as_deref().unwrap_or(""),
input.password.as_deref().unwrap_or(""),
);
match map_to_xtream_streams(xtream_cluster, streams, &input.name).await {
Ok(xtream_streams) => {
@@ -162,8 +166,10 @@ pub async fn parse_xtream(input: &ConfigInput,
channels: vec![],
};
let (live_stream_use_prefix, live_stream_without_extension) = input.options.as_ref()
.map_or((true, false), |o| (o.xtream_live_stream_use_prefix, o.xtream_live_stream_without_extension));
let (live_stream_use_prefix, live_stream_without_extension) = (
input.has_flag(ConfigInputFlags::XtreamLiveStreamUsePrefix),
input.has_flag(ConfigInputFlags::XtreamLiveStreamWithoutExtension),
);
// Re-implement the loop to add ordinal
let mut ord_counter: u32 = 1;
@@ -234,9 +240,19 @@ where
// 2. Prepare for Stream Parsing
let input_name = input.name.clone();
let url = input.url.as_str().to_string();
let username = input.username.as_ref().map_or("", |v| v).to_string();
let password = input.password.as_ref().map_or("", |v| v).to_string();
let options = input.options.clone();
let (username, password) = (
input.username.as_deref().unwrap_or("").to_string(),
input.password.as_deref().unwrap_or("").to_string(),
);
let (live_stream_use_prefix, live_stream_without_extension) = input.options.as_ref().map_or(
(false, false),
|o| {
(
o.has_flag(ConfigInputFlags::XtreamLiveStreamUsePrefix),
o.has_flag(ConfigInputFlags::XtreamLiveStreamWithoutExtension),
)
},
);
// Map categories for lookup
let group_map: IndexMap<u32, Arc<str>> = xtream_categories.iter().map(|c| (c.category_id, c.category_name.clone())).collect();
@@ -246,9 +262,6 @@ where
let reader = tokio_util::io::SyncIoBridge::new(streams);
let mut deserializer = serde_json::Deserializer::from_reader(reader);
let (live_stream_use_prefix, live_stream_without_extension) = options.as_ref()
.map_or((true, false), |o| (o.xtream_live_stream_use_prefix, o.xtream_live_stream_without_extension));
let mut source_ordinal = 0u32;
match xtream_cluster {
@@ -438,4 +451,4 @@ mod tests {
serde::Deserializer::deserialize_any(&mut deserializer, visitor).unwrap();
assert_eq!(count, 1);
}
}
}
backend/src/processing/processor/mod.rs
+56 -19
View File
@@ -14,20 +14,31 @@ pub use self::xtream::*;
pub use self::xtream_vod::*;
pub use self::xtream_series::*;
pub use self::stream_probe::*;
use shared::create_bit_set;
use shared::create_bitset;
create_bit_set!(u16, ResolveOptionsFlags, Resolve, TmdbMissing, Probe, Background);
create_bitset!(u8, ResolveOptionsFlags, Resolve, TmdbMissing, Probe, Background);
pub struct ResolveOptions {
pub flags: ResolveOptionsFlagsSet,
flags: ResolveOptionsFlagsSet,
pub resolve_delay: u16,
}
impl ResolveOptions {
#[inline]
pub fn has_flag(&self, flag: ResolveOptionsFlags) -> bool {
self.flags.contains(flag)
}
#[inline]
pub fn unset_flag(&mut self, flag: ResolveOptionsFlags) {
self.flags.unset(flag);
}
}
impl Default for ResolveOptions {
fn default() -> Self {
Self {
flags: ResolveOptionsFlagsSet::from_variants(&[ResolveOptionsFlags::Background]),
resolve_delay: 0,
resolve_delay: default_resolve_delay_secs(),
}
}
}
@@ -41,30 +52,55 @@ macro_rules! create_resolve_options_function_for_xtream_target {
($cluster:ident) => {
paste::paste! {
fn [<get_resolve_ $cluster _options>](target: &ConfigTarget, fpl: &FetchedPlaylist) -> $crate::processing::processor::ResolveOptions {
// Get input options
let (resolve_tmdb_missing, probe_requested) = fpl.input.options.as_ref().map_or_else(||(false, false), |o| (o.resolve_tmdb, o.probe_stream));
match target.get_xtream_output() {
Some(xtream_output) => {
Some(_) => {
let input_options = fpl.input.options.as_ref();
let input_is_xtream = fpl.input.input_type.is_xtream();
let (
resolve_tmdb_missing,
input_resolve_enabled,
input_probe_enabled,
input_resolve_delay,
resolve_background
) = if let Some(options) = input_options {
(
options.has_flag($crate::model::ConfigInputFlags::ResolveTmdb),
options.has_flag($crate::model::ConfigInputFlags::[<Resolve $cluster:camel>]),
options.has_flag($crate::model::ConfigInputFlags::[<Probe $cluster:camel>]),
options.resolve_delay,
options.has_flag($crate::model::ConfigInputFlags::ResolveBackground)
)
} else {
(
false,
false,
false,
shared::utils::default_resolve_delay_secs(),
shared::utils::default_resolve_background(),
)
};
let resolve_enabled = input_resolve_enabled;
let probe_enabled = input_probe_enabled;
let resolve_delay = input_resolve_delay;
let mut flags = $crate::processing::processor::ResolveOptionsFlagsSet::new();
// Map config flags to resolve options flags
if xtream_output.flags.contains($crate::model::XtreamTargetFlags::[<Resolve $cluster:camel>]) && fpl.input.input_type.is_xtream() {
flags.add($crate::processing::processor::ResolveOptionsFlags::Resolve);
if resolve_enabled && input_is_xtream {
flags.set($crate::processing::processor::ResolveOptionsFlags::Resolve);
}
if resolve_tmdb_missing {
flags.add($crate::processing::processor::ResolveOptionsFlags::TmdbMissing);
flags.set($crate::processing::processor::ResolveOptionsFlags::TmdbMissing);
}
if probe_requested
&& fpl.input.input_type.is_xtream()
&& xtream_output.flags.contains($crate::model::XtreamTargetFlags::[<Probe $cluster:camel>]) {
flags.add($crate::processing::processor::ResolveOptionsFlags::Probe);
if input_is_xtream && probe_enabled {
flags.set($crate::processing::processor::ResolveOptionsFlags::Probe);
}
if xtream_output.flags.contains($crate::model::XtreamTargetFlags::ResolveBackground) {
flags.add($crate::processing::processor::ResolveOptionsFlags::Background);
if resolve_background {
flags.set($crate::processing::processor::ResolveOptionsFlags::Background);
}
$crate::processing::processor::ResolveOptions {
flags,
resolve_delay: xtream_output.resolve_delay,
resolve_delay,
}
},
None => $crate::processing::processor::ResolveOptions::default(),
@@ -74,3 +110,4 @@ macro_rules! create_resolve_options_function_for_xtream_target {
};
}
use create_resolve_options_function_for_xtream_target;
use shared::utils::default_resolve_delay_secs;
backend/src/processing/processor/playlist.rs
+275 -192
View File
@@ -1,20 +1,24 @@
use crate::model::{AppConfig, ConfigFavourites, ConfigInput, ConfigRename, MessageContent, ReverseProxyDisabledHeaderConfig, TVGuide};
use crate::utils::{log_memory_snapshot, m3u};
use crate::model::{
AppConfig, ConfigFavourites, ConfigInput, ConfigRename, MessageContent, ReverseProxyDisabledHeaderConfig, TVGuide,
};
use crate::utils::xtream;
use crate::utils::{epg, StepMeasureCallback};
use crate::utils::{log_memory_snapshot, m3u};
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
use std::sync::{Arc, Weak};
use tokio::sync::{Mutex, OwnedRwLockWriteGuard, RwLock};
use tokio::task::JoinSet;
use crate::api::model::{ActiveProviderManager, EventManager, EventMessage, PlaylistStorageState, UpdateGuard, MetadataUpdateManager, ProviderIdType, ResolveReasonSet, ResolveReason, UpdateTask};
use crate::messaging::{send_message};
use crate::api::model::{
ActiveProviderManager, EventManager, EventMessage, MetadataUpdateManager, PlaylistStorageState, ProviderIdType,
ResolveReason, ResolveReasonSet, UpdateGuard, UpdateTask,
};
use crate::messaging::send_message;
use crate::model::FetchedPlaylist;
use crate::model::Mapping;
use crate::model::{ConfigTarget, ProcessTargets, TargetOutput, XtreamTargetFlags};
use crate::model::{ConfigInputFlags, ConfigInputOptions, ConfigTarget, ProcessTargets, TargetOutput};
use crate::processing::input_cache;
use crate::processing::input_cache::ClusterState;
use crate::processing::parser::xmltv::flatten_tvguide;
@@ -34,14 +38,18 @@ use indexmap::IndexMap;
use log::{debug, error, info, log_enabled, warn, Level};
use shared::concat_string;
use shared::error::{get_errors_notify_message, notify_err, TuliproxError};
use shared::foundation::{get_field_value, set_field_value, ValueAccessor, ValueProvider};
use shared::foundation::Filter;
use shared::foundation::{get_field_value, set_field_value, ValueAccessor, ValueProvider};
use shared::model::xtream_const::XTREAM_CLUSTER;
use shared::model::{
CounterModifier, FieldGetAccessor, FieldSetAccessor, InputType, ItemField, PlaylistGroup, PlaylistItem,
PlaylistItemType, PlaylistUpdateState, ProcessingOrder, StreamProperties, XtreamCluster,
};
use shared::model::{InputStats, PlaylistStats, SourceStats, TargetStats, UUIDType};
use shared::model::{CounterModifier, FieldGetAccessor, FieldSetAccessor, InputType, ItemField,
PlaylistGroup, PlaylistItem, PlaylistItemType, PlaylistUpdateState,
ProcessingOrder, StreamProperties, XtreamCluster};
use shared::utils::{create_alias_uuid, default_as_default, interner_gc, Internable};
use shared::utils::{
create_alias_uuid, default_as_default, default_probe_live_interval, default_resolve_delay_secs, interner_gc,
Internable,
};
use std::time::Instant;
const DEFAULT_PROBE_STREAM_DELAY: u16 = 50;
@@ -60,18 +68,24 @@ pub fn apply_filter_to_source(source: &mut dyn PlaylistSource, filter: &Filter)
let cat_id = pli.header.category_id;
let normalized_group = shared::utils::deunicode_string(&group_title).to_lowercase().intern();
let key = (cluster, normalized_group);
groups.entry(key)
groups
.entry(key)
.or_insert_with(|| PlaylistGroup {
id: cat_id,
title: group_title,
channels: vec![],
xtream_cluster: cluster,
})
.channels.push(pli);
.channels
.push(pli);
}
}
if groups.is_empty() { None } else { Some(groups.into_values().collect()) }
if groups.is_empty() {
None
} else {
Some(groups.into_values().collect())
}
}
fn filter_playlist(source: &mut dyn PlaylistSource, target: &ConfigTarget) -> Option<Vec<PlaylistGroup>> {
@@ -81,8 +95,8 @@ fn filter_playlist(source: &mut dyn PlaylistSource, target: &ConfigTarget) -> Op
pub fn apply_filter_to_playlist(playlist: &mut [PlaylistGroup], filter: &Filter) -> Option<Vec<PlaylistGroup>> {
let mut new_playlist = Vec::with_capacity(128);
for pg in playlist.iter_mut() {
let channels = pg.channels.iter()
.filter(|&pli| is_valid(pli, filter, false)).cloned().collect::<Vec<PlaylistItem>>();
let channels =
pg.channels.iter().filter(|&pli| is_valid(pli, filter, false)).cloned().collect::<Vec<PlaylistItem>>();
if !channels.is_empty() {
new_playlist.push(PlaylistGroup {
id: pg.id,
@@ -92,14 +106,16 @@ pub fn apply_filter_to_playlist(playlist: &mut [PlaylistGroup], filter: &Filter)
});
}
}
if new_playlist.is_empty() { None } else { Some(new_playlist) }
if new_playlist.is_empty() {
None
} else {
Some(new_playlist)
}
}
fn assign_channel_no_playlist(new_playlist: &mut [PlaylistGroup]) {
let assigned_chnos: HashSet<u32> = new_playlist.iter().flat_map(|g| &g.channels)
.filter(|c| c.header.chno != 0)
.map(|c| c.header.chno)
.collect();
let assigned_chnos: HashSet<u32> =
new_playlist.iter().flat_map(|g| &g.channels).filter(|c| c.header.chno != 0).map(|c| c.header.chno).collect();
let mut chno = 1;
for group in new_playlist {
for chan in &mut group.channels {
@@ -150,22 +166,23 @@ fn rename_playlist(source: &mut dyn PlaylistSource, target: &ConfigTarget) -> Op
let group_title = pli.header.group.clone();
let cluster = pli.header.xtream_cluster;
let cat_id = pli.header.category_id;
groups.entry((cluster, group_title.clone()))
groups
.entry((cluster, group_title.clone()))
.or_insert_with(|| PlaylistGroup {
id: cat_id,
title: group_title,
channels: vec![],
xtream_cluster: cluster,
})
.channels.push(pli);
.channels
.push(pli);
}
Some(groups.into_values().collect())
}
_ => None
_ => None,
}
}
fn map_channel(mut channel: PlaylistItem, mapping: &Mapping) -> (PlaylistItem, Vec<PlaylistItem>, bool) {
let mut matched = false;
let mut virtual_items = vec![];
@@ -179,7 +196,11 @@ fn map_channel(mut channel: PlaylistItem, mapping: &Mapping) -> (PlaylistItem, V
let provider = ValueProvider { pli: ref_chan, match_as_ascii: mapping.match_as_ascii };
if filter.filter(&provider) {
matched = true;
let mut accessor = ValueAccessor { pli: ref_chan, virtual_items: vec![], match_as_ascii: mapping.match_as_ascii };
let mut accessor = ValueAccessor {
pli: ref_chan,
virtual_items: vec![],
match_as_ascii: mapping.match_as_ascii,
};
script.eval(&mut accessor, templates);
virtual_items.extend(accessor.virtual_items.into_iter().map(|(_, pli)| pli));
}
@@ -214,17 +235,14 @@ fn map_playlist(source: &mut dyn PlaylistSource, target: &ConfigTarget) -> Optio
for channel in mapped_iter {
let group_title = channel.header.group.clone();
let cluster = channel.header.xtream_cluster;
next_groups.entry((cluster, group_title.clone()))
next_groups
.entry((cluster, group_title.clone()))
.or_insert_with(|| {
grp_id += 1;
PlaylistGroup {
id: grp_id,
title: group_title,
channels: Vec::new(),
xtream_cluster: cluster,
}
PlaylistGroup { id: grp_id, title: group_title, channels: Vec::new(), xtream_cluster: cluster }
})
.channels.push(channel);
.channels
.push(channel);
}
Some(next_groups.into_values().collect())
@@ -249,7 +267,10 @@ fn map_playlist_counter(target: &ConfigTarget, playlist: &mut [PlaylistGroup]) {
let new_value = if counter.modifier == CounterModifier::Assign {
padded_cntval
} else {
let value = channel.header.get_field(&counter.field).map_or_else(String::new, |field_value| field_value.to_string());
let value = channel
.header
.get_field(&counter.field)
.map_or_else(String::new, |field_value| field_value.to_string());
if counter.modifier == CounterModifier::Suffix {
format!("{value}{}{padded_cntval}", counter.concat)
} else {
@@ -277,7 +298,7 @@ fn is_target_enabled(target: &ConfigTarget, user_targets: &ProcessTargets) -> bo
}
struct PlaylistDownloadResult {
pub downloaded_playlist:Vec<PlaylistGroup>,
pub downloaded_playlist: Vec<PlaylistGroup>,
pub download_err: Vec<TuliproxError>,
pub was_cached: bool,
pub persisted: bool,
@@ -285,21 +306,20 @@ struct PlaylistDownloadResult {
impl PlaylistDownloadResult {
pub fn new(
downloaded_playlist:Vec<PlaylistGroup>,
downloaded_playlist: Vec<PlaylistGroup>,
download_err: Vec<TuliproxError>,
was_cached: bool,
persisted: bool) -> Self {
Self {
downloaded_playlist,
download_err,
was_cached,
persisted,
}
persisted: bool,
) -> Self {
Self { downloaded_playlist, download_err, was_cached, persisted }
}
}
async fn playlist_download_from_input(client: &reqwest::Client, app_config: &Arc<AppConfig>, input: &ConfigInput) -> PlaylistDownloadResult {
async fn playlist_download_from_input(
client: &reqwest::Client,
app_config: &Arc<AppConfig>,
input: &ConfigInput,
) -> PlaylistDownloadResult {
let config = &*app_config.config.load();
let working_dir = &config.working_dir;
@@ -346,7 +366,9 @@ async fn playlist_download_from_input(client: &reqwest::Client, app_config: &Arc
let (p, e) = m3u::download_m3u_playlist(app_config, client, config, input).await;
(p, e, false)
}
InputType::Xtream => xtream::download_xtream_playlist(app_config, client, input, clusters_to_download.as_deref()).await,
InputType::Xtream => {
xtream::download_xtream_playlist(app_config, client, input, clusters_to_download.as_deref()).await
}
InputType::M3uBatch | InputType::XtreamBatch => (vec![], vec![], false),
InputType::Library => {
let (p, e) = library::download_library_playlist(client, app_config, input).await;
@@ -391,7 +413,11 @@ async fn playlist_download_from_input(client: &reqwest::Client, app_config: &Arc
PlaylistDownloadResult::new(playlist, errors, false, persisted)
}
async fn process_source(source_idx: usize, ctx: &PlaylistProcessingContext) -> (Vec<InputStats>, Vec<TargetStats>, Vec<TuliproxError>) {
#[allow(clippy::too_many_lines)]
async fn process_source(
source_idx: usize,
ctx: &PlaylistProcessingContext,
) -> (Vec<InputStats>, Vec<TargetStats>, Vec<TuliproxError>) {
log_memory_snapshot(format!("source[{source_idx}] start").as_str());
let sources = ctx.config.sources.load();
let mut errors = vec![];
@@ -419,7 +445,10 @@ async fn process_source(source_idx: usize, ctx: &PlaylistProcessingContext) -> (
let (mut download_err, playlist, error) = download_input(ctx, input).await;
if let Some(err) = error {
broadcast_step("Playlist download", &format!("Failed to persist/load input '{}' playlist", input.name));
broadcast_step(
"Playlist download",
&format!("Failed to persist/load input '{}' playlist", input.name),
);
error!("Failed to persist input playlist {}", input.name);
download_err.push(err);
}
@@ -443,30 +472,32 @@ async fn process_source(source_idx: usize, ctx: &PlaylistProcessingContext) -> (
info!("Source is empty {input_name}");
errors.push(notify_err!("Source is empty {input_name}"));
} else {
source_playlists.push(
FetchedPlaylist {
input,
source: playlist_groups,
epg: tvguide,
}
source_playlists.push(FetchedPlaylist { input, source: playlist_groups, epg: tvguide });
log_memory_snapshot(
format!("source[{source_idx}] input '{}' after_source_push", input.name).as_str(),
);
log_memory_snapshot(format!("source[{source_idx}] input '{}' after_source_push", input.name).as_str());
}
let elapsed = start_time.elapsed().as_secs();
input_stats.insert(input_name.clone(), create_input_stat(group_count, channel_count, errors.len(),
input.input_type, input_name, elapsed));
input_stats.insert(
input_name.clone(),
create_input_stat(group_count, channel_count, errors.len(), input.input_type, input_name, elapsed),
);
}
}
if source_downloaded {
if source_playlists.is_empty() {
debug!("Source at index {source_idx} is empty");
errors.push(notify_err!("Source at index {source_idx} is empty: {}", source.inputs.iter().map(Clone::clone).collect::<Vec<Arc<str>>>().join(", ")));
errors.push(notify_err!(
"Source at index {source_idx} is empty: {}",
source.inputs.iter().map(Clone::clone).collect::<Vec<Arc<str>>>().join(", ")
));
} else {
debug_if_enabled!("Source has {} groups", source_playlists.iter_mut().map(FetchedPlaylist::get_channel_count).sum::<usize>());
let enabled_targets: Vec<_> = source.targets
.iter()
.filter(|target| is_target_enabled(target, &ctx.user_targets))
.collect();
debug_if_enabled!(
"Source has {} groups",
source_playlists.iter_mut().map(FetchedPlaylist::get_channel_count).sum::<usize>()
);
let enabled_targets: Vec<_> =
source.targets.iter().filter(|target| is_target_enabled(target, &ctx.user_targets)).collect();
for (idx, target) in enabled_targets.iter().enumerate() {
let consume_input_source = idx + 1 == enabled_targets.len();
@@ -474,9 +505,19 @@ async fn process_source(source_idx: usize, ctx: &PlaylistProcessingContext) -> (
"Processing target '{}' (use_memory_cache={}, consume_input_source={})",
target.name, target.use_memory_cache, consume_input_source
);
log_memory_snapshot(format!("source[{source_idx}] target '{}' before_process", target.name).as_str());
match process_playlist_for_target(ctx, &mut source_playlists, target,
&mut input_stats, &mut errors, consume_input_source).await {
log_memory_snapshot(
format!("source[{source_idx}] target '{}' before_process", target.name).as_str(),
);
match process_playlist_for_target(
ctx,
&mut source_playlists,
target,
&mut input_stats,
&mut errors,
consume_input_source,
)
.await
{
Ok(()) => {
target_stats.push(TargetStats::success(&target.name));
}
@@ -485,7 +526,9 @@ async fn process_source(source_idx: usize, ctx: &PlaylistProcessingContext) -> (
errors.append(&mut err);
}
}
log_memory_snapshot(format!("source[{source_idx}] target '{}' after_process", target.name).as_str());
log_memory_snapshot(
format!("source[{source_idx}] target '{}' after_process", target.name).as_str(),
);
}
}
}
@@ -494,8 +537,11 @@ async fn process_source(source_idx: usize, ctx: &PlaylistProcessingContext) -> (
(input_stats.into_values().collect(), target_stats, errors)
}
async fn download_input_epg(ctx: &PlaylistProcessingContext, input: &Arc<ConfigInput>,
error_list: &mut Vec<TuliproxError>) -> Option<TVGuide> {
async fn download_input_epg(
ctx: &PlaylistProcessingContext,
input: &Arc<ConfigInput>,
error_list: &mut Vec<TuliproxError>,
) -> Option<TVGuide> {
// Download epg for input
let (tvguide, mut tvguide_errors) = if error_list.is_empty() {
let working_dir = &ctx.config.config.load().working_dir;
@@ -507,12 +553,14 @@ async fn download_input_epg(ctx: &PlaylistProcessingContext, input: &Arc<ConfigI
tvguide
}
async fn download_input(ctx: &PlaylistProcessingContext, input: &Arc<ConfigInput>)
-> (Vec<TuliproxError>, Box<dyn PlaylistSource>, Option<TuliproxError>) {
async fn download_input(
ctx: &PlaylistProcessingContext,
input: &Arc<ConfigInput>,
) -> (Vec<TuliproxError>, Box<dyn PlaylistSource>, Option<TuliproxError>) {
// Coordination Logic
let need_download = !ctx.is_input_downloaded(&input.name).await;
let playlist_download_result= if need_download {
let playlist_download_result = if need_download {
// Acquire named lock to prevent thundering herd on same input
let _input_lock = ctx.get_input_lock(&input.name).await;
// Check again after lock
@@ -557,24 +605,24 @@ fn create_broadcast_callback(event_manager: Option<&Arc<EventManager>>) -> StepM
}
}
fn create_input_stat(group_count: usize, channel_count: usize, error_count: usize, input_type: InputType, input_name: &str, secs_took: u64) -> InputStats {
fn create_input_stat(
group_count: usize,
channel_count: usize,
error_count: usize,
input_type: InputType,
input_name: &str,
secs_took: u64,
) -> InputStats {
InputStats {
name: input_name.to_string(),
input_type,
error_count,
raw_stats: PlaylistStats {
group_count,
channel_count,
},
processed_stats: PlaylistStats {
group_count: 0,
channel_count: 0,
},
raw_stats: PlaylistStats { group_count, channel_count },
processed_stats: PlaylistStats { group_count: 0, channel_count: 0 },
secs_took,
}
}
#[derive(Clone)]
pub struct PlaylistProcessingContext {
pub client: reqwest::Client,
@@ -607,13 +655,11 @@ impl PlaylistProcessingContext {
pub async fn get_input_lock(&self, input_name: &Arc<str>) -> OwnedRwLockWriteGuard<()> {
let mut locks = self.input_locks.lock().await;
// Try to upgrade the existing weak reference
let lock = locks.get(input_name)
.and_then(Weak::upgrade)
.unwrap_or_else(|| {
let new_lock = Arc::new(RwLock::new(()));
locks.insert(input_name.clone(), Arc::downgrade(&new_lock));
new_lock
});
let lock = locks.get(input_name).and_then(Weak::upgrade).unwrap_or_else(|| {
let new_lock = Arc::new(RwLock::new(()));
locks.insert(input_name.clone(), Arc::downgrade(&new_lock));
new_lock
});
// Clean up stale references periodically
locks.retain(|_, weak| weak.strong_count() > 0);
@@ -654,8 +700,7 @@ async fn process_sources(processing_ctx: &PlaylistProcessingContext) -> (Vec<Sou
async_tasks.spawn(async move {
// Hold the per-source lock for the full duration of this update.
let current_update_lock = update_lock;
let (input_stats, target_stats, mut res_errors) =
process_source(index, &ctx).await;
let (input_stats, target_stats, mut res_errors) = process_source(index, &ctx).await;
shared_errors.lock().await.append(&mut res_errors);
if let Some(process_stats) = SourceStats::try_new(input_stats, target_stats) {
shared_stats.lock().await.push(process_stats);
@@ -663,8 +708,7 @@ async fn process_sources(processing_ctx: &PlaylistProcessingContext) -> (Vec<Sou
drop(current_update_lock);
});
} else {
let (input_stats, target_stats, mut res_errors) =
process_source(index, &ctx).await;
let (input_stats, target_stats, mut res_errors) = process_source(index, &ctx).await;
shared_errors.lock().await.append(&mut res_errors);
if let Some(process_stats) = SourceStats::try_new(input_stats, target_stats) {
shared_stats.lock().await.push(process_stats);
@@ -693,12 +737,17 @@ fn get_processing_pipe(target: &ConfigTarget) -> ProcessingPipe {
ProcessingOrder::Rfm => vec![rename_playlist, filter_playlist, map_playlist],
ProcessingOrder::Rmf => vec![rename_playlist, map_playlist, filter_playlist],
ProcessingOrder::Mfr => vec![map_playlist, filter_playlist, rename_playlist],
ProcessingOrder::Mrf => vec![map_playlist, rename_playlist, filter_playlist]
ProcessingOrder::Mrf => vec![map_playlist, rename_playlist, filter_playlist],
}
}
fn execute_pipe<'a>(target: &ConfigTarget, pipe: &ProcessingPipe, fpl: &mut FetchedPlaylist<'a>,
duplicates: &mut HashSet<UUIDType>, consume_source: bool) -> FetchedPlaylist<'a> {
fn execute_pipe<'a>(
target: &ConfigTarget,
pipe: &ProcessingPipe,
fpl: &mut FetchedPlaylist<'a>,
duplicates: &mut HashSet<UUIDType>,
consume_source: bool,
) -> FetchedPlaylist<'a> {
let source = if consume_source {
if fpl.is_memory() {
MemoryPlaylistSource::new(fpl.source.take_groups()).boxed()
@@ -709,11 +758,7 @@ fn execute_pipe<'a>(target: &ConfigTarget, pipe: &ProcessingPipe, fpl: &mut Fetc
fpl.clone_source()
};
let mut new_fpl = FetchedPlaylist {
input: fpl.input,
source,
epg: fpl.epg.clone(),
};
let mut new_fpl = FetchedPlaylist { input: fpl.input, source, epg: fpl.epg.clone() };
if target.options.as_ref().is_some_and(|opt| opt.remove_duplicates) {
new_fpl.deduplicate(duplicates);
}
@@ -756,12 +801,13 @@ fn flatten_groups(playlistgroups: Vec<PlaylistGroup>) -> Vec<PlaylistGroup> {
}
#[allow(clippy::too_many_arguments)]
async fn process_playlist_for_target(ctx: &PlaylistProcessingContext,
playlists: &mut [FetchedPlaylist<'_>],
target: &ConfigTarget,
stats: &mut HashMap<Arc<str>, InputStats>,
errors: &mut Vec<TuliproxError>,
consume_input_source: bool,
async fn process_playlist_for_target(
ctx: &PlaylistProcessingContext,
playlists: &mut [FetchedPlaylist<'_>],
target: &ConfigTarget,
stats: &mut HashMap<Arc<str>, InputStats>,
errors: &mut Vec<TuliproxError>,
consume_input_source: bool,
) -> Result<(), Vec<TuliproxError>> {
debug_if_enabled!("Processing order is {}", &target.processing_order);
log_memory_snapshot(format!("target '{}' start", target.name).as_str());
@@ -776,13 +822,19 @@ async fn process_playlist_for_target(ctx: &PlaylistProcessingContext,
let pipe = get_processing_pipe(target);
let mut step = StepMeasure::new(&target.name, broadcast_step);
for provider_fpl in playlists.iter_mut() {
log_memory_snapshot(format!("target '{}' input '{}' before_pipe", target.name, provider_fpl.input.name).as_str());
log_memory_snapshot(
format!("target '{}' input '{}' before_pipe", target.name, provider_fpl.input.name).as_str(),
);
step.broadcast("Executing transformations on '{}' playlist", &target.name);
let mut processed_fpl = execute_pipe(target, &pipe, provider_fpl, &mut duplicates, consume_input_source);
log_memory_snapshot(format!("target '{}' input '{}' after_pipe", target.name, provider_fpl.input.name).as_str());
log_memory_snapshot(
format!("target '{}' input '{}' after_pipe", target.name, provider_fpl.input.name).as_str(),
);
processed_fpl.sort_by_provider_ordinal();
playlist_resolve(ctx, target, errors, &pipe, provider_fpl, &mut processed_fpl).await;
log_memory_snapshot(format!("target '{}' input '{}' after_vod_resolve", target.name, provider_fpl.input.name).as_str());
log_memory_snapshot(
format!("target '{}' input '{}' after_vod_resolve", target.name, provider_fpl.input.name).as_str(),
);
// stats
let input_entry_name = processed_fpl.input.name.clone();
let group_count = processed_fpl.get_group_count();
@@ -792,9 +844,13 @@ async fn process_playlist_for_target(ctx: &PlaylistProcessingContext,
stat.processed_stats.channel_count = channel_count;
}
process_playlist_epg(&mut processed_fpl, &mut new_epg).await;
log_memory_snapshot(format!("target '{}' input '{}' after_epg_apply", target.name, processed_fpl.input.name).as_str());
log_memory_snapshot(
format!("target '{}' input '{}' after_epg_apply", target.name, processed_fpl.input.name).as_str(),
);
new_playlist.extend(processed_fpl.source.take_groups());
log_memory_snapshot(format!("target '{}' input '{}' after_take_groups", target.name, processed_fpl.input.name).as_str());
log_memory_snapshot(
format!("target '{}' input '{}' after_take_groups", target.name, processed_fpl.input.name).as_str(),
);
tokio::task::yield_now().await;
}
step.tick("filter rename map + epg");
@@ -836,78 +892,103 @@ async fn process_playlist_for_target(ctx: &PlaylistProcessingContext,
step.tick("group watches");
log_memory_snapshot(format!("target '{}' after_group_watches", target.name).as_str());
}
let result = persist_playlist(&ctx.config, &mut flat_new_playlist, flatten_tvguide(new_epg).as_ref(), target, ctx.playlist_state.as_ref()).await;
let result = persist_playlist(
&ctx.config,
&mut flat_new_playlist,
flatten_tvguide(new_epg).as_ref(),
target,
ctx.playlist_state.as_ref(),
)
.await;
step.stop("Persisting playlists");
log_memory_snapshot(format!("target '{}' after_persist", target.name).as_str());
result
}
}
async fn playlist_resolve(ctx: &PlaylistProcessingContext,
target: &ConfigTarget,
errors: &mut Vec<TuliproxError>,
pipe: &ProcessingPipe,
provider_fpl: &mut FetchedPlaylist<'_>,
processed_fpl: &mut FetchedPlaylist<'_>,
async fn playlist_resolve(
ctx: &PlaylistProcessingContext,
target: &ConfigTarget,
errors: &mut Vec<TuliproxError>,
pipe: &ProcessingPipe,
provider_fpl: &mut FetchedPlaylist<'_>,
processed_fpl: &mut FetchedPlaylist<'_>,
) {
playlist_resolve_series(ctx, target, errors, pipe, provider_fpl, processed_fpl).await;
playlist_resolve_vod(ctx, target, errors, provider_fpl, processed_fpl).await;
playlist_probe(ctx, target, processed_fpl).await;
}
fn is_xtream_only_output_target(target: &ConfigTarget) -> bool {
target.output.len() == 1 && matches!(target.output.first(), Some(TargetOutput::Xtream(_)))
fn has_target_m3u_output(target: &ConfigTarget) -> bool {
for output in &target.output {
if matches!(output, TargetOutput::M3u(_)) {
return true;
}
}
false
}
fn get_xtream_cluster_probe_flag(target: &ConfigTarget, cluster: XtreamCluster) -> Option<bool> {
target.get_xtream_output().map(|xtream_output| match cluster {
XtreamCluster::Live => xtream_output.flags.contains(XtreamTargetFlags::ProbeLive),
XtreamCluster::Video => xtream_output.flags.contains(XtreamTargetFlags::ProbeVod),
XtreamCluster::Series => xtream_output.flags.contains(XtreamTargetFlags::ProbeSeries),
fn get_xtream_cluster_probe_flag(
target: &ConfigTarget,
input_options: Option<&ConfigInputOptions>,
cluster: XtreamCluster,
) -> Option<bool> {
target.get_xtream_output().map(|_| match cluster {
XtreamCluster::Live => input_options.is_some_and(|options| options.has_flag(ConfigInputFlags::ProbeLive)),
XtreamCluster::Video => input_options.is_some_and(|options| options.has_flag(ConfigInputFlags::ProbeVod)),
XtreamCluster::Series => input_options.is_some_and(|options| options.has_flag(ConfigInputFlags::ProbeSeries)),
})
}
fn is_cluster_probe_enabled_for_target(target: &ConfigTarget, cluster: XtreamCluster) -> bool {
fn is_cluster_probe_enabled_for_target(
target: &ConfigTarget,
input_options: Option<&ConfigInputOptions>,
cluster: XtreamCluster,
) -> bool {
// Probe flags are only authoritative when the target is exactly one xtream output.
// For mixed outputs (xtream + m3u/strm/...), probing remains enabled because non-xtream outputs have no probe flags.
if !is_xtream_only_output_target(target) {
// For mixed outputs (xtream + m3u), probing remains enabled because non-xtream outputs have no probe flags.
if has_target_m3u_output(target) {
return true;
}
get_xtream_cluster_probe_flag(target, cluster).unwrap_or(true)
get_xtream_cluster_probe_flag(target, input_options, cluster).unwrap_or(true)
}
fn is_probe_supported_item_type(item_type: PlaylistItemType) -> bool {
matches!(
item_type,
PlaylistItemType::Live // we skip other live streams because hls and dash have multiple resolutions
item_type,
PlaylistItemType::Live // we skip other live streams because hls and dash have multiple resolutions
| PlaylistItemType::Video
| PlaylistItemType::LocalVideo
| PlaylistItemType::Series
| PlaylistItemType::LocalSeries
)
)
}
fn has_probe_details(item: &PlaylistItem) -> bool {
match item.header.additional_properties.as_ref() {
Some(StreamProperties::Video(v)) => v
.details
.as_ref()
.is_some_and(|d| d.video.is_some() && d.audio.is_some()),
Some(StreamProperties::Video(v)) => v.details.as_ref().is_some_and(|d| d.video.is_some() && d.audio.is_some()),
Some(StreamProperties::Live(l)) => l.video.is_some() && l.audio.is_some(),
Some(StreamProperties::Episode(e)) => e.video.is_some() && e.audio.is_some(),
Some(StreamProperties::Series(_)) | None => false,
}
}
fn get_live_probe_interval_settings(target: &ConfigTarget, input_type: InputType) -> Option<(u16, u64)> {
if !matches!(input_type, InputType::Xtream | InputType::XtreamBatch) {
fn get_live_probe_interval_settings(
target: &ConfigTarget,
input_type: InputType,
input_options: Option<&ConfigInputOptions>,
) -> Option<(u16, u64)> {
if !(input_type.is_xtream() || input_type.is_m3u()) {
return None;
}
target
.get_xtream_output()
.map(|xtream_output| (xtream_output.resolve_delay, u64::from(xtream_output.probe_live_interval_hours) * 3600))
target.get_xtream_output().map(|_| {
let (resolve_delay, input_probe_live_interval_hours) = input_options.map_or(
(default_resolve_delay_secs(), default_probe_live_interval()),
|options| (options.resolve_delay, options.probe_live_interval_hours),
);
(resolve_delay, u64::from(input_probe_live_interval_hours) * 3600)
})
}
fn needs_live_probe(item: &PlaylistItem, cutoff_ts: i64) -> bool {
@@ -946,7 +1027,7 @@ async fn playlist_probe(ctx: &PlaylistProcessingContext, target: &ConfigTarget,
let Some(opts) = fpl.input.options.as_ref() else {
return;
};
if !opts.probe_stream {
if !opts.has_any_flags(ConfigInputFlags::ProbeLive | ConfigInputFlags::ProbeVod | ConfigInputFlags::ProbeSeries) {
return;
}
if !ctx.config.is_ffprobe_enabled().await {
@@ -955,12 +1036,12 @@ async fn playlist_probe(ctx: &PlaylistProcessingContext, target: &ConfigTarget,
let input_name = fpl.input.name.clone();
let input_type = fpl.input.input_type;
let xtream_only_output = is_xtream_only_output_target(target);
let live_probe_settings = get_live_probe_interval_settings(target, input_type).map(|(delay, interval_secs)| {
let interval_signed = i64::try_from(interval_secs).unwrap_or(i64::MAX);
let cutoff_ts = chrono::Utc::now().timestamp().saturating_sub(interval_signed);
(delay, interval_secs, cutoff_ts)
});
let live_probe_settings =
get_live_probe_interval_settings(target, input_type, Some(opts)).map(|(delay, interval_secs)| {
let interval_signed = i64::try_from(interval_secs).unwrap_or(i64::MAX);
let cutoff_ts = chrono::Utc::now().timestamp().saturating_sub(interval_signed);
(delay, interval_secs, cutoff_ts)
});
let mut queued_probe_keys: HashSet<(Arc<str>, String)> = HashSet::new();
let mut queued_live_keys: HashSet<ProviderIdType> = HashSet::new();
@@ -968,7 +1049,7 @@ async fn playlist_probe(ctx: &PlaylistProcessingContext, target: &ConfigTarget,
let mut queued_stream_count = 0usize;
for item in fpl.items() {
if !is_cluster_probe_enabled_for_target(target, item.header.xtream_cluster) {
if !is_cluster_probe_enabled_for_target(target, Some(opts), item.header.xtream_cluster) {
continue;
}
@@ -996,26 +1077,18 @@ async fn playlist_probe(ctx: &PlaylistProcessingContext, target: &ConfigTarget,
}
}
// For xtream-only targets, dedicated xtream resolve/probe processors handle non-live paths.
if xtream_only_output && matches!(input_type, InputType::Xtream | InputType::XtreamBatch) {
continue;
}
if has_probe_details(&item) {
continue;
}
// For M3U, ID is provider id; for Library, ID is UUID.
// For M3U, ID is a provider id; for Library, ID is UUID.
let unique_id = if input_type == InputType::Library {
item.header.uuid.to_valid_uuid()
} else {
item.header.id.to_string()
};
let probe_scope = if item.header.input_name.is_empty() {
input_name.clone()
} else {
item.header.input_name.clone()
};
let probe_scope =
if item.header.input_name.is_empty() { input_name.clone() } else { item.header.input_name.clone() };
if !queued_probe_keys.insert((probe_scope.clone(), unique_id.clone())) {
continue;
@@ -1034,9 +1107,7 @@ async fn playlist_probe(ctx: &PlaylistProcessingContext, target: &ConfigTarget,
}
if queued_live_count > 0 || queued_stream_count > 0 {
info!(
"Queued probe tasks for input {input_name} (live_interval={queued_live_count}, generic={queued_stream_count})"
);
info!("Queued probe tasks for input {input_name} (live_interval={queued_live_count}, generic={queued_stream_count})");
}
}
@@ -1046,7 +1117,9 @@ pub fn process_favourites(playlist: &mut Vec<PlaylistGroup>, favourites_cfg: Opt
for pg in playlist.iter() {
for pli in &pg.channels {
// series episodes can't be included in favourites
if pli.header.item_type == PlaylistItemType::Series || pli.header.item_type == PlaylistItemType::LocalSeries {
if pli.header.item_type == PlaylistItemType::Series
|| pli.header.item_type == PlaylistItemType::LocalSeries
{
continue;
}
for fav in favourites {
@@ -1055,10 +1128,7 @@ pub fn process_favourites(playlist: &mut Vec<PlaylistGroup>, favourites_cfg: Opt
channel.header.group.clone_from(&fav.group);
// Update UUID to be an alias of the original
channel.header.uuid = create_alias_uuid(&pli.header.uuid, &fav.group);
fav_groups
.entry((fav.cluster, fav.group.clone()))
.or_default()
.push(channel);
fav_groups.entry((fav.cluster, fav.group.clone())).or_default().push(channel);
}
}
}
@@ -1067,18 +1137,18 @@ pub fn process_favourites(playlist: &mut Vec<PlaylistGroup>, favourites_cfg: Opt
for (fav_group, channels) in fav_groups {
if !channels.is_empty() {
let (xtream_cluster, group_name) = fav_group;
playlist.push(PlaylistGroup {
id: 0,
title: group_name,
channels,
xtream_cluster,
});
playlist.push(PlaylistGroup { id: 0, title: group_name, channels, xtream_cluster });
}
}
}
}
async fn trakt_playlist(client: &reqwest::Client, target: &ConfigTarget, errors: &mut Vec<TuliproxError>, playlist: &mut Vec<PlaylistGroup>) -> bool {
async fn trakt_playlist(
client: &reqwest::Client,
target: &ConfigTarget,
errors: &mut Vec<TuliproxError>,
playlist: &mut Vec<PlaylistGroup>,
) -> bool {
match process_trakt_categories_for_target(client, playlist, target).await {
Ok(Some(trakt_categories)) => {
if !trakt_categories.is_empty() {
@@ -1097,7 +1167,12 @@ async fn trakt_playlist(client: &reqwest::Client, target: &ConfigTarget, errors:
true
}
async fn process_watch(app_config: &Arc<AppConfig>, client: &reqwest::Client, target: &ConfigTarget, new_playlist: &[PlaylistGroup]) -> bool {
async fn process_watch(
app_config: &Arc<AppConfig>,
client: &reqwest::Client,
target: &ConfigTarget,
new_playlist: &[PlaylistGroup],
) -> bool {
if let Some(watches) = &target.watch {
if default_as_default().eq_ignore_ascii_case(&target.name) {
error!("can't watch a target with no unique name");
@@ -1108,8 +1183,10 @@ async fn process_watch(app_config: &Arc<AppConfig>, client: &reqwest::Client, ta
new_playlist
.iter()
.filter(|pl| watches.iter().any(|r| r.is_match(&pl.title)))
.map(|pl| process_group_watch(app_config, client, &target.name, pl))
).for_each_concurrent(16, |f| f).await;
.map(|pl| process_group_watch(app_config, client, &target.name, pl)),
)
.for_each_concurrent(16, |f| f)
.await;
true
} else {
@@ -1118,12 +1195,18 @@ async fn process_watch(app_config: &Arc<AppConfig>, client: &reqwest::Client, ta
}
#[allow(clippy::too_many_arguments)]
pub async fn exec_processing(client: &reqwest::Client, app_config: Arc<AppConfig>, targets: Arc<ProcessTargets>,
event_manager: Option<Arc<EventManager>>, playlist_state: Option<Arc<PlaylistStorageState>>,
update_guard: Option<UpdateGuard>, disabled_headers: Option<ReverseProxyDisabledHeaderConfig>,
provider_manager: Option<Arc<ActiveProviderManager>>,
metadata_manager: Option<Arc<MetadataUpdateManager>>) {
let _guard = if let Some(guard) = update_guard {
pub async fn exec_processing(
client: &reqwest::Client,
app_config: Arc<AppConfig>,
targets: Arc<ProcessTargets>,
event_manager: Option<Arc<EventManager>>,
playlist_state: Option<Arc<PlaylistStorageState>>,
update_guard: Option<UpdateGuard>,
disabled_headers: Option<ReverseProxyDisabledHeaderConfig>,
provider_manager: Option<Arc<ActiveProviderManager>>,
metadata_manager: Option<Arc<MetadataUpdateManager>>,
) {
let _guard = if let Some(guard) = update_guard {
if let Some(permit) = guard.try_playlist() {
Some(permit)
} else {
@@ -1184,7 +1267,10 @@ pub async fn exec_processing(client: &reqwest::Client, app_config: Arc<AppConfig
let update_finished_message = format!("🌷 Update process finished! Took {elapsed} secs.");
if let Some(events) = &event_manager {
events.send_event(EventMessage::PlaylistUpdateProgress("Playlist Update".to_string(), update_finished_message.clone()));
events.send_event(EventMessage::PlaylistUpdateProgress(
"Playlist Update".to_string(),
update_finished_message.clone(),
));
}
log_memory_snapshot("exec_processing before_interner_gc");
debug!("StringInterner GC removed {} strings", interner_gc());
@@ -1200,10 +1286,7 @@ mod tests {
use shared::utils::Internable;
fn item_with_props(props: StreamProperties) -> PlaylistItem {
let header = shared::model::PlaylistItemHeader {
additional_properties: Some(props),
..Default::default()
};
let header = shared::model::PlaylistItemHeader { additional_properties: Some(props), ..Default::default() };
PlaylistItem { header }
}
backend/src/processing/processor/xtream.rs
+3 -4
View File
@@ -1,6 +1,6 @@
use std::sync::Arc;
use shared::error::{TuliproxError};
use crate::model::{AppConfig, ConfigInput};
use crate::model::{AppConfig, ConfigInput, ConfigInputFlags};
use shared::model::{LiveStreamProperties, StreamProperties, XtreamCluster, XtreamPlaylistItem};
use crate::repository::{get_input_storage_path, persist_input_live_info, BPlusTreeQuery, xtream_get_file_path};
use crate::utils::{debug_if_enabled};
@@ -71,9 +71,8 @@ pub async fn update_live_stream_metadata(
let input_url = input.url.as_str();
let username = input.username.as_deref().unwrap_or("");
let password = input.password.as_deref().unwrap_or("");
let opts = input.options.as_ref();
let use_prefix = opts.is_none_or(|o| o.xtream_live_stream_use_prefix);
let no_ext = opts.is_some_and(|o| o.xtream_live_stream_without_extension);
let use_prefix = input.has_flag(ConfigInputFlags::XtreamLiveStreamUsePrefix);
let no_ext = input.has_flag(ConfigInputFlags::XtreamLiveStreamWithoutExtension);
// We generate the URL to probe directly on the provider
let stream_url = create_xtream_url(
backend/src/processing/processor/xtream_series.rs
+36 -32
View File
@@ -3,7 +3,7 @@ use crate::api::model::{ActiveProviderManager, ProviderHandle, ProviderIdType, R
use crate::library::MetadataResolver;
use crate::model::FetchedPlaylist;
use crate::model::{AppConfig, ConfigTarget};
use crate::model::{ConfigInput, InputSource};
use crate::model::{ConfigInput, ConfigInputFlags, InputSource};
use crate::processing::parser::xtream::create_xtream_series_episode_url;
use crate::processing::parser::xtream::parse_xtream_series_info;
use crate::processing::processor::playlist::{PlaylistProcessingContext, ProcessingPipe};
@@ -37,10 +37,16 @@ pub async fn playlist_resolve_series(
provider_fpl: &mut FetchedPlaylist<'_>,
processed_fpl: &mut FetchedPlaylist<'_>,
) {
let resolve_options = get_resolve_series_options(target, processed_fpl);
// Skip-flag is the kill-switch: no resolve, no probe, no iteration.
if processed_fpl.input.has_flag(ConfigInputFlags::XtreamSkipSeries) {
return;
}
let do_probe = resolve_options.flags.contains(ResolveOptionsFlags::Probe) && ctx.config.is_ffprobe_enabled().await;
let mut resolve_options = get_resolve_series_options(target, processed_fpl);
if !ctx.config.is_ffprobe_enabled().await {
resolve_options.unset_flag(ResolveOptionsFlags::Probe);
}
provider_fpl.source.release_resources(XtreamCluster::Series);
playlist_resolve_series_info(
@@ -48,7 +54,6 @@ pub async fn playlist_resolve_series(
errors,
processed_fpl,
resolve_options,
do_probe,
pipe,
target,
)
@@ -66,7 +71,6 @@ async fn playlist_resolve_series_info(ctx: &PlaylistProcessingContext,
_errors: &mut Vec<TuliproxError>,
fpl: &mut FetchedPlaylist<'_>,
resolve_options: ResolveOptions,
do_probe: bool,
pipe: &ProcessingPipe,
target: &ConfigTarget) {
let filter = |pli: &PlaylistItem| {
@@ -79,17 +83,18 @@ async fn playlist_resolve_series_info(ctx: &PlaylistProcessingContext,
};
// Skip if nothing to do
let skip_resolve = !resolve_options.flags.contains(ResolveOptionsFlags::Resolve) && !do_probe && !resolve_options.flags.contains(ResolveOptionsFlags::TmdbMissing);
let skip_resolve = !resolve_options.has_flag(ResolveOptionsFlags::Resolve)
&& !resolve_options.has_flag(ResolveOptionsFlags::Probe)
&& !resolve_options.has_flag(ResolveOptionsFlags::TmdbMissing);
let resolve_tmdb_enabled = fpl.input.options.as_ref().is_some_and(|o| o.resolve_tmdb);
let resolve_tmdb_enabled = fpl.input.has_flag(ConfigInputFlags::ResolveTmdb);
let groups_to_add = if resolve_options.flags.contains(ResolveOptionsFlags::Background) && ctx.metadata_manager.is_some() {
let groups_to_add = if resolve_options.has_flag(ResolveOptionsFlags::Background) && ctx.metadata_manager.is_some() {
queue_background_series_info(
ctx,
fpl,
filter,
&resolve_options,
do_probe,
skip_resolve,
resolve_tmdb_enabled,
)
@@ -99,7 +104,6 @@ async fn playlist_resolve_series_info(ctx: &PlaylistProcessingContext,
fpl,
filter,
&resolve_options,
do_probe,
skip_resolve,
resolve_tmdb_enabled,
)
@@ -180,7 +184,6 @@ fn queue_background_series_info(
fpl: &mut FetchedPlaylist<'_>,
filter: impl Fn(&PlaylistItem) -> bool,
resolve_options: &ResolveOptions,
do_probe: bool,
skip_resolve: bool,
resolve_tmdb_enabled: bool,
) -> Vec<PlaylistGroup> {
@@ -204,7 +207,7 @@ fn queue_background_series_info(
};
if !skip_resolve {
let reasons = check_resolve_reasons(resolve_options, do_probe, resolve_tmdb_enabled, pli);
let reasons = check_resolve_reasons(resolve_options, resolve_tmdb_enabled, pli);
if !reasons.is_empty() {
let task = UpdateTask::ResolveSeries {
@@ -229,7 +232,6 @@ async fn process_immediate_series_info(
fpl: &mut FetchedPlaylist<'_>,
filter: impl Fn(&PlaylistItem) -> bool,
resolve_options: &ResolveOptions,
do_probe: bool,
skip_resolve: bool,
resolve_tmdb_enabled: bool,
) -> Vec<PlaylistGroup> {
@@ -265,7 +267,7 @@ async fn process_immediate_series_info(
};
if !skip_resolve {
let reasons = check_resolve_reasons(resolve_options, do_probe, resolve_tmdb_enabled, pli);
let reasons = check_resolve_reasons(resolve_options, resolve_tmdb_enabled, pli);
if !reasons.is_empty() {
if let Some(active_provider) = ctx.provider_manager.as_ref() {
@@ -420,7 +422,6 @@ async fn update_series_info_immediate(
fn check_resolve_reasons(
resolve_options: &ResolveOptions,
do_probe: bool,
resolve_tmdb_enabled: bool,
pli: &mut PlaylistItem,
) -> ResolveReasonSet {
@@ -429,16 +430,16 @@ fn check_resolve_reasons(
let needs_info = check_needs_info(resolve_options, pli, &mut reasons);
check_resolve_tmdb(resolve_options, resolve_tmdb_enabled, pli, needs_info, &mut reasons);
if do_probe {
check_needs_probe(pli, &mut reasons);
if resolve_options.has_flag(ResolveOptionsFlags::Probe) {
check_needs_probe(pli, needs_info, &mut reasons);
}
reasons
}
fn check_needs_info(resolve_options: &ResolveOptions, pli: &mut PlaylistItem, reasons: &mut ResolveReasonSet) -> bool {
let needs_info = resolve_options.flags.contains(ResolveOptionsFlags::Resolve) && !pli.has_details();
let needs_info = resolve_options.has_flag(ResolveOptionsFlags::Resolve) && !pli.has_details();
if needs_info {
reasons.add(ResolveReason::Info);
reasons.set(ResolveReason::Info);
}
needs_info
}
@@ -450,25 +451,26 @@ fn check_resolve_tmdb(
needs_info: bool,
reasons: &mut ResolveReasonSet,
) {
if resolve_tmdb_enabled && resolve_options.flags.contains(ResolveOptionsFlags::TmdbMissing) {
if resolve_tmdb_enabled
&& resolve_options.has_flag(ResolveOptionsFlags::TmdbMissing)
&& needs_info {
if let Some(StreamProperties::Series(series_stream_props)) = pli.header.additional_properties.as_ref() {
let has_tmdb = series_stream_props.tmdb.is_some();
let has_date = series_stream_props.release_date.is_some();
let title_present = !series_stream_props.name.is_empty() || !pli.header.title.is_empty();
if title_present && (!has_tmdb || !has_date) {
if !has_tmdb { reasons.add(ResolveReason::Tmdb); }
if !has_date { reasons.add(ResolveReason::Date); }
if !has_tmdb { reasons.set(ResolveReason::Tmdb); }
if !has_date { reasons.set(ResolveReason::Date); }
}
} else if needs_info {
// Covered by info
} else {
reasons.add(ResolveReason::MissingDetails);
}
}
}
fn check_needs_probe(pli: &mut PlaylistItem, reasons: &mut ResolveReasonSet) {
fn check_needs_probe(pli: &mut PlaylistItem, needs_info: bool, reasons: &mut ResolveReasonSet) {
if !needs_info {
return;
}
if let Some(StreamProperties::Series(series_stream_props)) = pli.header.additional_properties.as_ref() {
if let Some(details) = series_stream_props.details.as_ref() {
if let Some(episodes) = details.episodes.as_ref() {
@@ -476,7 +478,7 @@ fn check_needs_probe(pli: &mut PlaylistItem, reasons: &mut ResolveReasonSet) {
let missing_video = !MediaQuality::is_valid_media_info(ep.video.as_deref());
let missing_audio = !MediaQuality::is_valid_media_info(ep.audio.as_deref());
if missing_video || missing_audio {
reasons.add(ResolveReason::Probe);
reasons.set(ResolveReason::Probe);
break;
}
}
@@ -510,8 +512,7 @@ pub async fn update_series_metadata(
let storage_path = get_input_storage_path(&input.name, working_dir).await
.map_err(|e| shared::error::info_err!("Storage path error: {e}"))?;
let opts = input.options.as_ref();
if opts.is_some_and(|o| o.xtream_skip_series) {
if input.has_flag(ConfigInputFlags::XtreamSkipSeries) {
return Ok(None);
}
@@ -600,10 +601,13 @@ pub async fn update_series_metadata(
let mut properties = props.unwrap();
let resolve_tmdb_enabled = input.options.as_ref().is_some_and(|o| o.resolve_tmdb);
let resolve_tmdb_enabled = input.has_flag(ConfigInputFlags::ResolveTmdb);
// 2. Resolve TMDB/Date if missing
if resolve_tmdb_enabled && (properties.tmdb.is_none() || properties.release_date.is_none()) && !properties.name.is_empty() {
if fetch_info
&& resolve_tmdb_enabled
&& (properties.tmdb.is_none() || properties.release_date.is_none())
&& !properties.name.is_empty() {
let config = app_config.config.load();
let library_config = config.library.as_ref();
let meta_resolver = MetadataResolver::new(library_config, client.clone());
backend/src/processing/processor/xtream_vod.rs
+42 -27
View File
@@ -1,7 +1,7 @@
use crate::api::model::{ActiveProviderManager, ProviderHandle};
use crate::api::model::{ResolveReason, ResolveReasonSet, UpdateTask, ProviderIdType};
use crate::library::MetadataResolver;
use crate::model::ConfigInput;
use crate::model::{ConfigInput, ConfigInputFlags};
use crate::model::FetchedPlaylist;
use crate::model::InputSource;
use crate::model::{AppConfig, ConfigTarget};
@@ -33,13 +33,20 @@ pub async fn playlist_resolve_vod(
provider_fpl: &mut FetchedPlaylist<'_>,
fpl: &mut FetchedPlaylist<'_>,
) {
// Skip-flag is the kill-switch: no resolve, no probe, no iteration.
if fpl.input.has_flag(ConfigInputFlags::XtreamSkipVod) {
return;
}
let resolve_options = get_resolve_vod_options(target, fpl);
let app_config: &Arc<AppConfig> = &ctx.config;
let do_probe = resolve_options.flags.contains(ResolveOptionsFlags::Probe) && app_config.is_ffprobe_enabled().await;
let do_probe = resolve_options.has_flag(ResolveOptionsFlags::Probe) && app_config.is_ffprobe_enabled().await;
// Determine if we need to do anything
if !resolve_options.flags.contains(ResolveOptionsFlags::Resolve) && !do_probe && !resolve_options.flags.contains(ResolveOptionsFlags::TmdbMissing) {
if !resolve_options.has_flag(ResolveOptionsFlags::Resolve)
&& !do_probe
&& !resolve_options.has_flag(ResolveOptionsFlags::TmdbMissing) {
return;
}
@@ -130,9 +137,9 @@ async fn playlist_resolve_vod_info(
true
};
let resolve_tmdb_enabled = fpl.input.options.as_ref().is_some_and(|o| o.resolve_tmdb);
let resolve_tmdb_enabled = fpl.input.has_flag(ConfigInputFlags::ResolveTmdb);
if resolve_options.flags.contains(ResolveOptionsFlags::Background) && ctx.metadata_manager.is_some() {
if resolve_options.has_flag(ResolveOptionsFlags::Background) && ctx.metadata_manager.is_some() {
queue_background_vod_info(
ctx,
fpl,
@@ -227,20 +234,25 @@ async fn process_immediate_vod_info(ctx: &PlaylistProcessingContext, fpl: &mut F
})
.collect();
if !updates.is_empty() {
if let Err(err) = persist_input_vod_info_batch(
if updates.is_empty() {
batch.clear();
} else {
match persist_input_vod_info_batch(
&ctx.config,
&storage_path,
XtreamCluster::Video,
&input.name,
updates,
).await {
error!("Failed to persist batch VOD info: {err}");
Ok(()) => batch.clear(),
Err(err) => {
error!(
"persist_input_vod_info_batch failed for XtreamCluster::Video on input '{}'. batch.clear() skipped. Error: {err}",
input.name
);
}
}
}
// Clear batch
batch.clear();
}
processed_count += 1;
@@ -301,12 +313,15 @@ fn check_resolve_reasons(
// Probe check
if do_probe {
check_needs_probe(pli, &mut reasons);
check_needs_probe(pli, needs_info, &mut reasons);
}
reasons
}
fn check_needs_probe(pli: &PlaylistItem, reasons: &mut ResolveReasonSet) {
fn check_needs_probe(pli: &PlaylistItem, needs_info: bool, reasons: &mut ResolveReasonSet) {
if !needs_info {
return;
}
let needs_probe = match pli.header.additional_properties.as_ref() {
Some(StreamProperties::Video(props)) => {
let details = props.details.as_ref();
@@ -320,15 +335,15 @@ fn check_needs_probe(pli: &PlaylistItem, reasons: &mut ResolveReasonSet) {
_ => false,
};
if needs_probe {
reasons.add(ResolveReason::Probe);
reasons.set(ResolveReason::Probe);
}
}
fn check_needs_info(resolve_options: &ResolveOptions, pli: &PlaylistItem, reasons: &mut ResolveReasonSet) -> bool {
let needs_info = !pli.has_details() && resolve_options.flags.contains(ResolveOptionsFlags::Resolve);
let needs_info = !pli.has_details() && resolve_options.has_flag(ResolveOptionsFlags::Resolve);
if needs_info {
reasons.add(ResolveReason::Info);
reasons.set(ResolveReason::Info);
}
needs_info
}
@@ -340,17 +355,16 @@ fn check_resolve_tmdb(
needs_info: bool,
reasons: &mut ResolveReasonSet,
) {
if resolve_tmdb_enabled && resolve_options.flags.contains(ResolveOptionsFlags::TmdbMissing) {
if resolve_tmdb_enabled
&& resolve_options.has_flag(ResolveOptionsFlags::TmdbMissing)
&& needs_info {
if let Some(StreamProperties::Video(video_stream_props)) = pli.header.additional_properties.as_ref() {
let has_tmdb = video_stream_props.tmdb.is_some();
let has_date = video_stream_props.details.as_ref().and_then(|d| d.release_date.as_ref()).is_some();
if !has_tmdb || !has_date {
if !has_tmdb { reasons.add(ResolveReason::Tmdb); }
if !has_date { reasons.add(ResolveReason::Date); }
if !has_tmdb { reasons.set(ResolveReason::Tmdb); }
if !has_date { reasons.set(ResolveReason::Date); }
}
} else if !needs_info {
// No properties but no download requested? Should be rare
reasons.add(ResolveReason::MissingDetails);
}
}
}
@@ -410,6 +424,7 @@ async fn update_vod_info_immediate(
Some(&pli.header.title),
false,
fetch_info,
reasons.contains(ResolveReason::Probe),
db_query,
).await
}
@@ -433,6 +448,7 @@ pub async fn update_vod_metadata(
playlist_title: Option<&str>,
save: bool,
fetch_info: bool,
do_probe: bool,
db_query: Option<&mut BPlusTreeQuery<u32, XtreamPlaylistItem>>,
) -> Result<Option<VideoStreamProperties>, TuliproxError> {
let working_dir = &app_config.config.load().working_dir;
@@ -440,8 +456,7 @@ pub async fn update_vod_metadata(
.map_err(|e| shared::error::info_err!("Storage path error: {e}"))?;
// Check if we should skip based on input options
let opts = input.options.as_ref();
if opts.is_some_and(|o| o.xtream_skip_vod) {
if input.has_flag(ConfigInputFlags::XtreamSkipVod) {
return Ok(None);
}
@@ -548,13 +563,13 @@ pub async fn update_vod_metadata(
// Now it's safe to unwrap because we handled the None case above
let mut properties = props.unwrap();
let resolve_tmdb_enabled = input.options.as_ref().is_some_and(|o| o.resolve_tmdb);
let resolve_tmdb_enabled = input.has_flag(ConfigInputFlags::ResolveTmdb);
// 2. Resolve TMDB/Date if missing and explicitly enabled for this input
let missing_tmdb = properties.tmdb.is_none();
let missing_date = properties.details.as_ref().and_then(|d| d.release_date.as_ref()).is_none();
if resolve_tmdb_enabled && (missing_tmdb || missing_date) {
if fetch_info && resolve_tmdb_enabled && (missing_tmdb || missing_date) {
// Try local parsing first
if missing_date && !properties.name.is_empty() {
let meta_parse = ptt_parse_title(&properties.name);
@@ -633,7 +648,7 @@ pub async fn update_vod_metadata(
// 3. Probe (if enabled globally in config)
let ffprobe_enabled = app_config.is_ffprobe_enabled().await;
if ffprobe_enabled {
if do_probe && ffprobe_enabled {
// Ensure details struct exists before probing
if properties.details.is_none() {
properties.details = Some(VideoStreamDetailProperties::default());
backend/src/ptt/handlers.rs
+341 -341
View File
File diff suppressed because it is too large Load Diff
backend/src/ptt/parser.rs
+63 -13
View File
@@ -3,6 +3,7 @@ use std::collections::HashMap;
use fancy_regex::{Regex as FancyRegex, Match};
use crate::ptt::constants::{BRACKETS, PTT_CONSTANTS};
use crate::ptt::models::PttMetadata;
use shared::create_bitset;
#[derive(Debug, Clone)]
pub struct MatchInfo {
@@ -20,6 +21,15 @@ pub struct ParseContext {
pub type HandlerFn = Box<dyn Fn(&mut ParseContext) -> Option<MatchInfo> + Send + Sync>;
create_bitset!(
u8,
HandlerOptionFlags,
SkipIfAlreadyFound,
SkipFromTitle,
SkipIfFirst,
Remove
);
#[derive(Default)]
pub struct PttParser {
handlers: Vec<(String, HandlerFn)>,
@@ -51,7 +61,11 @@ impl PttParser {
let name_clone = name.to_string();
let handler = Box::new(move |context: &mut ParseContext| -> Option<MatchInfo> {
if options.skip_if_already_found && context.matched.contains_key(&name_owned) {
if options
.flags
.contains(HandlerOptionFlags::SkipIfAlreadyFound)
&& context.matched.contains_key(&name_owned)
{
return None;
}
@@ -85,11 +99,12 @@ impl PttParser {
false
};
let current_skip_from_title = before_title_matched || options.skip_from_title;
let current_skip_from_title = before_title_matched
|| options.flags.contains(HandlerOptionFlags::SkipFromTitle);
let transformed_value = transformer(&clean_match);
if options.skip_if_first {
if options.flags.contains(HandlerOptionFlags::SkipIfFirst) {
let other_matches_exist = !context.matched.is_empty();
if other_matches_exist {
let current_start = match_index;
@@ -108,7 +123,7 @@ impl PttParser {
let info = MatchInfo {
raw_match,
match_index,
remove: options.remove,
remove: options.flags.contains(HandlerOptionFlags::Remove),
skip_from_title: current_skip_from_title,
};
context.matched.insert(name_owned.clone(), info.clone());
@@ -229,21 +244,56 @@ pub fn clean_title(raw_title: &str) -> String {
title
}
#[allow(clippy::struct_excessive_bools)]
pub struct HandlerOptions {
pub skip_if_already_found: bool,
pub skip_from_title: bool,
pub skip_if_first: bool,
pub remove: bool,
pub flags: HandlerOptionFlagsSet,
}
impl HandlerOptions {
pub fn set_flag(&mut self, flag: HandlerOptionFlags, enabled: bool) {
if enabled {
self.flags.set(flag);
} else {
self.flags.unset(flag);
}
}
}
impl Default for HandlerOptions {
fn default() -> Self {
let mut flags = HandlerOptionFlagsSet::new();
flags.set(HandlerOptionFlags::SkipIfAlreadyFound);
Self {
skip_if_already_found: true,
skip_from_title: false,
skip_if_first: false,
remove: false,
flags,
}
}
}
macro_rules! handler_options_set_field {
($opts:ident, skip_if_already_found, $value:expr) => {
$opts.set_flag($crate::ptt::parser::HandlerOptionFlags::SkipIfAlreadyFound, $value);
};
($opts:ident, skip_from_title, $value:expr) => {
$opts.set_flag($crate::ptt::parser::HandlerOptionFlags::SkipFromTitle, $value);
};
($opts:ident, skip_if_first, $value:expr) => {
$opts.set_flag($crate::ptt::parser::HandlerOptionFlags::SkipIfFirst, $value);
};
($opts:ident, remove, $value:expr) => {
$opts.set_flag($crate::ptt::parser::HandlerOptionFlags::Remove, $value);
};
}
pub(crate) use handler_options_set_field;
macro_rules! handler_options {
($($field:ident : $value:expr),* , ..Default::default() $(,)?) => {{
let mut options = $crate::ptt::parser::HandlerOptions::default();
$($crate::ptt::parser::handler_options_set_field!(options, $field, $value);)*
options
}};
($($field:ident : $value:expr),* $(,)?) => {{
let mut options = $crate::ptt::parser::HandlerOptions::default();
$($crate::ptt::parser::handler_options_set_field!(options, $field, $value);)*
options
}};
}
pub(crate) use handler_options;
backend/src/repository/m3u_playlist_iterator.rs
+53 -23
View File
@@ -2,6 +2,7 @@ use shared::error::info_err;
use shared::error::{TuliproxError};
use crate::model::{AppConfig, ProxyUserCredentials};
use crate::model::{ConfigTarget};
use shared::create_bitset;
use shared::model::{ConfigTargetOptions, M3uPlaylistItem, PlaylistItemType, ProxyType, TargetType, XtreamCluster};
use crate::repository::{BPlusTreeQuery, PlaylistIteratorReader};
use crate::repository::m3u_get_file_path_for_db;
@@ -14,22 +15,33 @@ use std::iter::Peekable;
use log::error;
use shared::utils::{extract_extension_from_url, Internable};
#[allow(clippy::struct_excessive_bools)]
pub struct M3uPlaylistIterator {
create_bitset!(
u8,
M3uPlaylistIteratorFlags,
MaskRedirectUrl,
IncludeTypeInUrl,
RewriteResource
);
struct M3uPlaylistCursor {
reader: Peekable<PlaylistIteratorReader<M3uPlaylistItem>>,
lookup_item: Option<M3uPlaylistItem>,
}
pub struct M3uPlaylistIterator {
cursor: Box<M3uPlaylistCursor>,
base_url: String,
username: String,
password: String,
target_options: Option<ConfigTargetOptions>,
mask_redirect_url: bool,
include_type_in_url: bool,
rewrite_resource: bool,
flags: M3uPlaylistIteratorFlagsSet,
proxy_type: ProxyType,
filter: Option<HashSet<String>>,
lookup_item: Option<(M3uPlaylistItem, bool)>,
_file_lock: FileReadGuard,
}
impl M3uPlaylistIterator {
pub async fn new(
cfg: &AppConfig,
@@ -66,21 +78,31 @@ impl M3uPlaylistIterator {
}.peekable();
let filter = user_get_bouquet_filter(&config, &user.username, None, TargetType::M3u, XtreamCluster::Live).await;
let mut flags = M3uPlaylistIteratorFlagsSet::new();
if m3u_output.include_type_in_url {
flags.set(M3uPlaylistIteratorFlags::IncludeTypeInUrl);
}
if m3u_output.mask_redirect_url {
flags.set(M3uPlaylistIteratorFlags::MaskRedirectUrl);
}
if cfg.is_reverse_proxy_resource_rewrite_enabled() {
flags.set(M3uPlaylistIteratorFlags::RewriteResource);
}
let server_info = cfg.get_user_server_info(user);
Ok(Self {
reader,
cursor: Box::new(M3uPlaylistCursor {
reader,
lookup_item: None,
}),
base_url: server_info.get_base_url(),
username: user.username.clone(),
password: user.password.clone(),
target_options: target.options.clone(),
include_type_in_url: m3u_output.include_type_in_url,
mask_redirect_url: m3u_output.mask_redirect_url,
flags,
filter,
proxy_type: user.proxy,
_file_lock: file_lock, // Save lock inside struct
rewrite_resource: cfg.is_reverse_proxy_resource_rewrite_enabled(),
lookup_item: None,
})
}
@@ -150,26 +172,26 @@ impl M3uPlaylistIterator {
fn get_next(&mut self) -> Option<(M3uPlaylistItem, bool)> {
let entry = if let Some(set) = &self.filter {
if let Some((current_item, _)) = self.lookup_item.take() {
if let Some(current_item) = self.cursor.lookup_item.take() {
// Avoid cloning strings while filtering
let next_valid = Self::find_next_matching(&mut self.reader, set);
self.lookup_item = next_valid.map(|v| (v, true)); // has_next handled by iterator usually, but here we just need the item
let has_next = self.lookup_item.is_some();
let next_valid = Self::find_next_matching(&mut self.cursor.reader, set);
self.cursor.lookup_item = next_valid;
let has_next = self.cursor.lookup_item.is_some();
Some((current_item, has_next))
} else {
let current_item = Self::find_next_matching(&mut self.reader, set);
let current_item = Self::find_next_matching(&mut self.cursor.reader, set);
if let Some(item) = current_item {
self.lookup_item = Self::find_next_matching(&mut self.reader, set).map(|item| (item, true));
let has_next = self.lookup_item.is_some();
self.cursor.lookup_item = Self::find_next_matching(&mut self.cursor.reader, set);
let has_next = self.cursor.lookup_item.is_some();
Some((item, has_next))
} else {
None
}
}
} else {
match self.reader.next() {
match self.cursor.reader.next() {
Some(Ok((_, v))) => {
let has_next = matches!(self.reader.peek(), Some(Ok(_)));
let has_next = matches!(self.cursor.reader.peek(), Some(Ok(_)));
Some((v, has_next))
},
Some(Err(e)) => {
@@ -188,11 +210,19 @@ impl M3uPlaylistIterator {
.as_ref()
.and_then(|o| o.force_redirect.as_ref())
.is_some_and(|f| f.has_cluster(m3u_pli.item_type));
let should_rewrite_urls = if is_redirect { self.mask_redirect_url } else { true };
let should_rewrite_urls = if is_redirect {
self.flags.contains(M3uPlaylistIteratorFlags::MaskRedirectUrl)
} else {
true
};
if should_rewrite_urls {
let stream_url = self.get_stream_url(&m3u_pli, self.include_type_in_url);
let resource_url = if self.rewrite_resource {
let stream_url = self.get_stream_url(
&m3u_pli,
self.flags
.contains(M3uPlaylistIteratorFlags::IncludeTypeInUrl),
);
let resource_url = if self.flags.contains(M3uPlaylistIteratorFlags::RewriteResource) {
Some(self.get_resource_url(&m3u_pli))
} else {
None
backend/src/repository/strm_repository.rs
+37 -9
View File
@@ -1,5 +1,5 @@
use crate::model::{ApiProxyServerInfo, AppConfig, ProxyUserCredentials};
use crate::model::{ConfigTarget, StrmTargetOutput};
use crate::model::{ConfigTarget, StrmTargetFlags, StrmTargetOutput};
use crate::repository::storage::ensure_target_storage_path;
use crate::repository::storage_const;
use crate::utils::{async_file_reader, async_file_writer, normalize_string_path, truncate_filename,
@@ -688,17 +688,29 @@ fn prepare_strm_files(
&strm_item_info,
pli.get_tmdb_id(),
strm_target_output.style,
strm_target_output.underscore_whitespace,
strm_target_output.flat,
strm_target_output
.flags
.contains(StrmTargetFlags::UnderscoreWhitespace),
strm_target_output.flags.contains(StrmTargetFlags::Flat),
&mut flat_dedup_paths,
);
// Conditionally generate the quality string based on the new config flag
let separator = if strm_target_output.underscore_whitespace { "_" } else { " " };
let separator = if strm_target_output
.flags
.contains(StrmTargetFlags::UnderscoreWhitespace)
{
"_"
} else {
" "
};
let quality_string = get_quality(strm_target_output, pli, separator);
// Add category suffix for flat movie structure to avoid collisions
let category_suffix = if strm_target_output.flat && pli.get_tmdb_id().is_some() && pli.header.item_type == PlaylistItemType::Video {
let category_suffix = if strm_target_output.flags.contains(StrmTargetFlags::Flat)
&& pli.get_tmdb_id().is_some()
&& pli.header.item_type == PlaylistItemType::Video
{
let cat = sanitize_for_filename(&strm_item_info.group, false);
format!("{separator}[{cat}]")
} else {
@@ -726,7 +738,14 @@ fn prepare_strm_files(
if !collisions.is_empty() {
// This separator is specifically for the multi-version naming convention.
let version_separator = " ";
let separator = if strm_target_output.underscore_whitespace { "_" } else { " " };
let separator = if strm_target_output
.flags
.contains(StrmTargetFlags::UnderscoreWhitespace)
{
"_"
} else {
" "
};
for s in &mut result {
{
@@ -750,7 +769,10 @@ fn prepare_strm_files(
}
fn get_quality(strm_target_output: &StrmTargetOutput, pli: &PlaylistItem, separator: &str) -> String {
if strm_target_output.add_quality_to_filename {
if strm_target_output
.flags
.contains(StrmTargetFlags::AddQualityToFilename)
{
// Use `additional_properties` which are populated by metadata_update_manager/probe
let (audio, video) = match pli.header.additional_properties.as_ref() {
None => (None, None),
@@ -867,7 +889,13 @@ pub async fn write_strm_playlist(
}
if let Err(err) =
cleanup_strm_output_directory(target_output.cleanup, &root_path, &existing_strm, &processed_strm).await
cleanup_strm_output_directory(
target_output.flags.contains(StrmTargetFlags::Cleanup),
&root_path,
&existing_strm,
&processed_strm,
)
.await
{
failed.push(err);
}
@@ -1154,4 +1182,4 @@ async fn delete_empty_dirs_from_tree(root_path: &Path, tree_nodes: HashMap<PathB
async fn remove_empty_dirs(root_path: PathBuf) {
let tree_nodes = build_directory_tree(&root_path).await;
delete_empty_dirs_from_tree(&root_path, tree_nodes).await;
}
}
backend/src/repository/xtream_repository.rs
+16 -13
View File
@@ -18,7 +18,7 @@ use bytes::Bytes;
use fs2::FileExt;
use futures::{stream, Stream, StreamExt};
use indexmap::IndexMap;
use log::error;
use log::{error};
use serde::{Deserialize, Serialize};
use serde_json::{json, Value};
use shared::error::{info_err_res, notify_err, string_to_io_error, TuliproxError};
@@ -718,9 +718,10 @@ fn preserve_details_input_xtream_playlist_cluster_to_disk(
old_path: &Path,
tmp_path: &Path,
) -> Result<(), TuliproxError> {
let Ok(mut old_tree) = BPlusTreeQuery::<u32, XtreamPlaylistItem>::try_new(old_path) else {
return Ok(())
};
let Ok(mut old_tree) = BPlusTreeQuery::<u32, XtreamPlaylistItem>::try_new(old_path) else {
return Ok(())
};
let Ok(mut new_tree) = BPlusTreeUpdate::<u32, XtreamPlaylistItem>::try_new_with_backoff(tmp_path) else {
return Ok(())
@@ -732,7 +733,8 @@ fn preserve_details_input_xtream_playlist_cluster_to_disk(
if old_props.has_details() {
if let Ok(Some(mut new_item)) = new_tree.query(&old_item.provider_id) {
if let Some(new_props) = new_item.additional_properties.as_mut() {
if needs_preserved_stream_property_merge(new_props, old_props) && merge_preserved_stream_properties(new_props, old_props) {
if needs_preserved_stream_property_merge(new_props, old_props)
&& merge_preserved_stream_properties(new_props, old_props) {
updates.push((new_item.provider_id, new_item));
if updates.len() >= BATCH_SIZE {
let refs: Vec<(&u32, &XtreamPlaylistItem)> = updates.iter().map(|(id, pli)| (id, pli)).collect();
@@ -1126,6 +1128,7 @@ pub(crate) fn needs_update_info_details(
new_stream_props: &StreamProperties,
old_stream_props: &StreamProperties,
) -> bool {
let new_modified = new_stream_props.get_last_modified();
let old_modified = old_stream_props.get_last_modified();
@@ -1144,15 +1147,15 @@ fn needs_preserved_stream_property_merge(
match (new_stream_props, old_stream_props) {
(StreamProperties::Video(v_new), StreamProperties::Video(v_old))
if preserve_info_details && v_old.details.is_some() =>
{
v_new.details != v_old.details
}
if preserve_info_details && v_old.details.is_some() =>
{
v_new.details != v_old.details
}
(StreamProperties::Series(s_new), StreamProperties::Series(s_old))
if preserve_info_details && s_old.details.is_some() =>
{
s_new.details != s_old.details
}
if preserve_info_details && s_old.details.is_some() =>
{
s_new.details != s_old.details
}
(StreamProperties::Live(l_new), StreamProperties::Live(l_old)) => {
(l_new.video.is_none() && l_old.video.is_some())
|| (l_new.audio.is_none() && l_old.audio.is_some())
backend/src/utils/network/request.rs
+3 -2
View File
@@ -200,8 +200,9 @@ pub async fn send_with_retry_and_provider(
// Record the starting URL index for full-cycle detection.
// This allows us to try all URLs even when starting from a non-zero index.
let start_index = provider.as_ref().map_or(0, |p| p.get_current_index());
let max_provider_attempts = provider.as_ref().map_or(0, |p| p.urls.len());
let (start_index, max_provider_attempts) = provider
.as_ref()
.map_or((0, 0), |p| (p.get_current_index(), p.urls.len()));
let mut provider_attempts = usize::from(max_provider_attempts > 0);
'provider_loop: loop {
backend/src/utils/network/xtream.rs
+18 -18
View File
@@ -1,7 +1,7 @@
use crate::api::model::AppState;
use crate::messaging::send_message;
use crate::model::{is_input_expired, xtream_mapping_option_from_target_options, AppConfig,
ConfigInput, ConfigTarget, MessageContent, XtreamLoginInfo, XtreamTargetOutput};
ConfigInput, ConfigInputFlags, ConfigTarget, MessageContent, XtreamLoginInfo, XtreamTargetOutput};
use crate::model::{InputSource, ProxyUserCredentials};
use crate::processing::parser::xtream;
use crate::processing::parser::xtream::parse_xtream_series_info;
@@ -23,7 +23,7 @@ use std::str::FromStr;
use std::sync::Arc;
use shared::info_err;
use shared::{concat_string, info_err};
const THREE_DAYS_IN_SECS: i64 = 3 * 24 * 60 * 60;
@@ -245,16 +245,14 @@ fn xtream_resolve_stream_info(app_state: &Arc<AppState>, user: &ProxyUserCredent
fn get_skip_cluster(input: &ConfigInput) -> Vec<XtreamCluster> {
let mut skip_cluster = vec![];
if let Some(input_options) = &input.options {
if input_options.xtream_skip_live {
skip_cluster.push(XtreamCluster::Live);
}
if input_options.xtream_skip_vod {
skip_cluster.push(XtreamCluster::Video);
}
if input_options.xtream_skip_series {
skip_cluster.push(XtreamCluster::Series);
}
if input.has_flag(ConfigInputFlags::XtreamSkipLive) {
skip_cluster.push(XtreamCluster::Live);
}
if input.has_flag(ConfigInputFlags::XtreamSkipVod) {
skip_cluster.push(XtreamCluster::Video);
}
if input.has_flag(ConfigInputFlags::XtreamSkipSeries) {
skip_cluster.push(XtreamCluster::Series);
}
if skip_cluster.len() == 3 {
info!("You have skipped all sections from xtream input {}", &input.name);
@@ -346,8 +344,10 @@ pub async fn download_xtream_playlist(app_config: &Arc<AppConfig>, client: &reqw
}
};
let username = input_source.username.as_ref().map_or("", |v| v);
let password = input_source.password.as_ref().map_or("", |v| v);
let (username, password) = (
input_source.username.as_deref().unwrap_or(""),
input_source.password.as_deref().unwrap_or(""),
);
let is_provider_url = input_source.url.starts_with(PROVIDER_SCHEME_PREFIX);
let base_input_url = if is_provider_url {
@@ -383,12 +383,12 @@ pub async fn download_xtream_playlist(app_config: &Arc<AppConfig>, client: &reqw
for (xtream_cluster, category, stream) in &ACTIONS {
let is_requested = clusters.is_none_or(|c| c.contains(xtream_cluster));
if is_requested && !skip_cluster.contains(xtream_cluster) {
let input_source_category = input_source.with_url(format!("{base_url}&action={category}"));
let input_source_stream = input_source.with_url(format!("{base_url}&action={stream}"));
let input_source_category = input_source.with_url(concat_string!(&base_url, "&action=", category));
let input_source_stream = input_source.with_url(concat_string!(&base_url, "&action=", stream));
let category_file_path = crate::utils::prepare_file_path(input.persist.as_deref(),
working_dir, format!("{category}_").as_str());
working_dir, concat_string!(category, "_").as_str());
let stream_file_path = crate::utils::prepare_file_path(input.persist.as_deref(),
working_dir, format!("{stream}_").as_str());
working_dir, concat_string!(stream, "_").as_str());
match futures::join!(
request::get_input_json_content_as_stream(app_config, client, &input_source_category, category_file_path),
config/source.yml
-2
View File
@@ -33,8 +33,6 @@ sources:
skip_live_direct_source: true
skip_video_direct_source: true
skip_series_direct_source: true
resolve_series: false
resolve_vod: false
trakt:
lists:
- user: "linaspurinis"
frontend/Cargo.toml
+2 -2
View File
@@ -29,11 +29,11 @@ futures = "0.3"
prost = "0"
wasm-bindgen-futures = "0"
bytes = "1"
regex = "1.12.2"
regex = "1.12.3"
base64 = "0.22.1"
cron = "0.15"
fastrand = "2.3.0"
indexmap = "2.12.1"
indexmap = "2.13.0"
[dependencies.web-sys]
version = "0.3"
frontend/public/assets/i18n/en.json
+3 -2
View File
@@ -222,8 +222,9 @@
"PATTERN": "Pattern",
"NEW_NAME": "New Name",
"SKIP": "Skip",
"LIVE_STREAM_USE_PREFIX": "Live use prefix",
"LIVE_STREAM_WITHOUT_EXTENSION": "Live w/o ext.",
"LIVE_STREAMS": "Live streams",
"LIVE_STREAM_USE_PREFIX": "Use prefix",
"LIVE_STREAM_WITHOUT_EXTENSION": "Without ext.",
"RESOLVE_TMDB": "Resolve TMDB",
"PROBE_STREAM": "Probe Stream",
"ALIAS": "Alias",
frontend/scss/app/components/_collapse_panel.scss
+1 -1
View File
@@ -18,7 +18,7 @@
&:hover {
color: var(--menu-item-hover-color);
fill: var(--menu-item-hover-color);
background-color: var(--menu-item-hover-background-color);
font-weight: bold;
}
&-title {
flex: 1 1 auto;
frontend/scss/app/components/source_editor/_source_editor_block.scss
+5
View File
@@ -17,6 +17,11 @@ $port-size: 10px;
width: $block-width;
box-sizing: border-box;
/* Safari Fixes */
will-change: transform;
-webkit-transform-style: preserve-3d;
backface-visibility: hidden;
&-editing {
.tp__source-editor__block-content {
border-color: red !important;
frontend/scss/app/components/source_editor/_source_editor_form.scss
+2 -2
View File
@@ -20,8 +20,8 @@ $form-width: 480px;
}
.tp__source-editor-form {
padding: 0 var(--padding-micro);
padding: 0 var(--padding-mini) var(--padding-mini) var(--padding-mini);
border-radius: var(--border-radius);
background-color: var(--card-background-color);
&__body {
frontend/src/app/components/playlist/input/input_options.rs
+82 -8
View File
@@ -1,5 +1,8 @@
use crate::app::components::{make_tags, CollapsePanel, Tag, TagList};
use shared::model::ConfigInputDto;
use crate::app::components::{chip::convert_bool_to_chip_style, make_tags, CollapsePanel, Tag, TagList};
use shared::{
model::ConfigInputDto,
utils::{default_probe_live_interval, default_resolve_delay_secs},
};
use std::rc::Rc;
use yew::prelude::*;
use yew_i18n::use_translation;
@@ -13,7 +16,7 @@ pub struct InputOptionsProps {
pub fn InputOptions(props: &InputOptionsProps) -> Html {
let translate = use_translation();
let tags = use_memo(props.input.clone(), |input| {
let (has_options, options1, options2) = match input.options.as_ref() {
let (has_options, options1, options2, resolve_tags, probe_tags) = match input.options.as_ref() {
None => (
false,
vec![(false, "LABEL.LIVE"), (false, "LABEL.VOD"), (false, "LABEL.SERIES")],
@@ -21,17 +24,51 @@ pub fn InputOptions(props: &InputOptionsProps) -> Html {
(false, "LABEL.LIVE_STREAM_USE_PREFIX"),
(false, "LABEL.LIVE_STREAM_WITHOUT_EXTENSION"),
(false, "LABEL.RESOLVE_TMDB"),
(false, "LABEL.PROBE_STREAM"),
],
vec![
Rc::new(Tag {
class: convert_bool_to_chip_style(false),
label: format!("{} / {}s", translate.t("LABEL.SERIES"), default_resolve_delay_secs()),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(false),
label: format!("{} / {}s", translate.t("LABEL.VOD"), default_resolve_delay_secs()),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(true),
label: translate.t("LABEL.RESOLVE_BACKGROUND").to_string(),
}),
],
vec![
Rc::new(Tag {
class: convert_bool_to_chip_style(false),
label: format!("{} / {}h", translate.t("LABEL.LIVE"), default_probe_live_interval()),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(false),
label: translate.t("LABEL.VOD").to_string(),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(false),
label: translate.t("LABEL.SERIES").to_string(),
}),
],
),
Some(options) => {
let has_options = options.xtream_skip_live
|| options.xtream_skip_vod
|| options.xtream_skip_series
|| options.xtream_live_stream_use_prefix
|| !options.xtream_live_stream_use_prefix
|| options.xtream_live_stream_without_extension
|| options.resolve_tmdb
|| options.probe_stream;
|| !options.resolve_background
|| options.resolve_series
|| options.resolve_vod
|| options.probe_series
|| options.probe_vod
|| options.probe_live
|| options.resolve_delay != default_resolve_delay_secs()
|| options.probe_live_interval_hours != default_probe_live_interval();
(
has_options,
@@ -44,17 +81,46 @@ pub fn InputOptions(props: &InputOptionsProps) -> Html {
(options.xtream_live_stream_use_prefix, "LABEL.LIVE_STREAM_USE_PREFIX"),
(options.xtream_live_stream_without_extension, "LABEL.LIVE_STREAM_WITHOUT_EXTENSION"),
(options.resolve_tmdb, "LABEL.RESOLVE_TMDB"),
(options.probe_stream, "LABEL.PROBE_STREAM"),
],
vec![
Rc::new(Tag {
class: convert_bool_to_chip_style(options.resolve_series),
label: format!("{} / {}s", translate.t("LABEL.SERIES"), options.resolve_delay),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(options.resolve_vod),
label: format!("{} / {}s", translate.t("LABEL.VOD"), options.resolve_delay),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(options.resolve_background),
label: translate.t("LABEL.RESOLVE_BACKGROUND").to_string(),
}),
],
vec![
Rc::new(Tag {
class: convert_bool_to_chip_style(options.probe_live),
label: format!("{} / {}h", translate.t("LABEL.LIVE"), options.probe_live_interval_hours),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(options.probe_vod),
label: translate.t("LABEL.VOD").to_string(),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(options.probe_series),
label: translate.t("LABEL.SERIES").to_string(),
}),
],
)
}
};
(has_options, make_tags(&options1, &translate), make_tags(&options2, &translate))
(has_options, make_tags(&options1, &translate), make_tags(&options2, &translate), resolve_tags, probe_tags)
});
let has_options = tags.0;
let opts1: Vec<Rc<Tag>> = tags.1.clone();
let opts2: Vec<Rc<Tag>> = tags.2.clone();
let resolve: Vec<Rc<Tag>> = tags.3.clone();
let probe: Vec<Rc<Tag>> = tags.4.clone();
html! {
<CollapsePanel expanded={false} class={format!("tp__target-options__panel{}", if has_options { " tp__target-options__has_options"} else {""})}
@@ -63,6 +129,14 @@ pub fn InputOptions(props: &InputOptionsProps) -> Html {
<div class="tp__target-options__section">
<TagList tags={opts2} />
</div>
<div class="tp__target-options__section">
<span class="tp__target-common__label">{translate.t("LABEL.RESOLVE")}</span>
<TagList tags={resolve} />
</div>
<div class="tp__target-options__section">
<span class="tp__target-common__label">{translate.t("LABEL.PROBE")}</span>
<TagList tags={probe} />
</div>
<div class="tp__target-options__section">
<span class="tp__target-common__label">{translate.t("LABEL.SKIP")}</span>
<TagList tags={opts1} />
frontend/src/app/components/playlist/target/xtream_output.rs
-45
View File
@@ -36,43 +36,6 @@ pub fn XtreamOutput(props: &XtreamOutputProps) -> Html {
]
})
};
let tags_resolve = {
let output = props.output.clone();
let translate = translator.clone();
use_memo(output, move |output| {
vec![
Rc::new(Tag {
class: convert_bool_to_chip_style(output.resolve_series),
label: format!("{} / {}ms", translate.t("LABEL.SERIES"), output.resolve_delay),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(output.resolve_vod),
label: format!("{} / {}ms", translate.t("LABEL.VOD"), output.resolve_delay),
}),
]
})
};
let tags_probe = {
let output = props.output.clone();
let translate = translator.clone();
use_memo(output, move |output| {
vec![
Rc::new(Tag {
class: convert_bool_to_chip_style(output.probe_live),
label: format!("{} / {}h", translate.t("LABEL.LIVE"), output.probe_live_interval_hours),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(output.probe_vod),
label: translate.t("LABEL.VOD").to_string(),
}),
Rc::new(Tag {
class: convert_bool_to_chip_style(output.probe_series),
label: translate.t("LABEL.SERIES").to_string(),
}),
]
})
};
html! {
<div class="tp__xtream-output tp__target-common">
{ html_if!(props.output.t_filter.is_some(), {
@@ -86,14 +49,6 @@ pub fn XtreamOutput(props: &XtreamOutputProps) -> Html {
<span class="tp__target-common__label">{translator.t("LABEL.SKIP_DIRECT_SOURCE")}</span>
<TagList tags={(*tags_skip_direct_source).clone()} />
</div>
<div class="tp__target-common__section">
<span class="tp__target-common__label">{translator.t("LABEL.RESOLVE")}</span>
<TagList tags={(*tags_resolve).clone()} />
</div>
<div class="tp__target-common__section">
<span class="tp__target-common__label">{translator.t("LABEL.PROBE")}</span>
<TagList tags={(*tags_probe).clone()} />
</div>
</div>
}
}
frontend/src/app/components/source_editor/editor_view.rs
+1 -1
View File
@@ -779,7 +779,7 @@ pub fn SourceEditor() -> Html {
div.style()
.set_property(
"transform",
&format!("translate({}px,{}px)", x + canvas_ox, y + canvas_oy),
&format!("translate3d({}px,{}px, 0)", x + canvas_ox, y + canvas_oy),
)
.unwrap();
}
frontend/src/app/components/source_editor/input_form.rs
+43 -10
View File
@@ -5,7 +5,7 @@ use crate::{
SourceEditorContext, TextButton, TitledCard,
},
config_field_child, edit_field_bool, edit_field_date, edit_field_number_i16, edit_field_number_u16,
edit_field_text, edit_field_text_option, generate_form_reducer, html_if,
edit_field_number_u32, edit_field_text, edit_field_text_option, generate_form_reducer, html_if,
};
use shared::{
error::TuliproxError,
@@ -46,15 +46,19 @@ const LABEL_XTREAM_SKIP_SERIES: &str = "LABEL.SERIES";
const LABEL_XTREAM_LIVE_STREAM_USE_PREFIX: &str = "LABEL.LIVE_STREAM_USE_PREFIX";
const LABEL_XTREAM_LIVE_STREAM_WITHOUT_EXTENSION: &str = "LABEL.LIVE_STREAM_WITHOUT_EXTENSION";
const LABEL_RESOLVE_TMDB: &str = "LABEL.RESOLVE_TMDB";
const LABEL_PROBE_STREAM: &str = "LABEL.PROBE_STREAM";
const LABEL_RESOLVE: &str = "LABEL.RESOLVE";
const LABEL_PROBE: &str = "LABEL.PROBE";
const LABEL_RESOLVE_DELAY_SEC: &str = "LABEL.RESOLVE_DELAY_SEC";
const LABEL_RESOLVE_BACKGROUND: &str = "LABEL.RESOLVE_BACKGROUND";
const LABEL_PROBE_LIVE_INTERVAL_HOURS: &str = "LABEL.PROBE_LIVE_INTERVAL_HOURS";
const LABEL_METADATA: &str = "LABEL.METADATA";
const LABEL_CACHE_DURATION: &str = "LABEL.CACHE_DURATION";
const LABEL_MAIN: &str = "LABEL.MAIN_CONFIG";
const LABEL_OPTIONS: &str = "LABEL.OPTIONS";
const LABEL_STAGED: &str = "LABEL.STAGED";
const LABEL_ADVANCED: &str = "LABEL.ADVANCED";
const LABEL_ALIAS: &str = "LABEL.ALIAS";
const LABEL_LIVE_STREAMS: &str = "LABEL.LIVE_STREAMS";
#[derive(Copy, Clone, PartialEq, Eq)]
enum InputFormPage {
@@ -114,7 +118,14 @@ generate_form_reducer!(
XtreamLiveStreamUsePrefix => xtream_live_stream_use_prefix: bool,
XtreamLiveStreamWithoutExtension => xtream_live_stream_without_extension: bool,
ResolveTmdb => resolve_tmdb: bool,
ProbeStream => probe_stream: bool,
ResolveBackground => resolve_background: bool,
ResolveSeries => resolve_series: bool,
ResolveVod => resolve_vod: bool,
ResolveDelay => resolve_delay: u16,
ProbeLive => probe_live: bool,
ProbeVod => probe_vod: bool,
ProbeSeries => probe_series: bool,
ProbeLiveIntervalHours => probe_live_interval_hours: u32,
}
);
@@ -418,15 +429,37 @@ pub fn ConfigInputView(props: &ConfigInputViewProps) -> Html {
{ edit_field_bool!(input_options_state, translate.t(LABEL_XTREAM_SKIP_SERIES), xtream_skip_series, ConfigInputOptionsFormAction::XtreamSkipSeries) }
</div>
</TitledCard>
{ edit_field_bool!(input_options_state, translate.t(LABEL_XTREAM_LIVE_STREAM_USE_PREFIX), xtream_live_stream_use_prefix, ConfigInputOptionsFormAction::XtreamLiveStreamUsePrefix) }
{ edit_field_bool!(input_options_state, translate.t(LABEL_XTREAM_LIVE_STREAM_WITHOUT_EXTENSION), xtream_live_stream_without_extension, ConfigInputOptionsFormAction::XtreamLiveStreamWithoutExtension) }
<TitledCard title={translate.t(LABEL_LIVE_STREAMS)}>
<div class="tp__config-view__cols-2">
{ edit_field_bool!(input_options_state, translate.t(LABEL_XTREAM_LIVE_STREAM_USE_PREFIX), xtream_live_stream_use_prefix, ConfigInputOptionsFormAction::XtreamLiveStreamUsePrefix) }
{ edit_field_bool!(input_options_state, translate.t(LABEL_XTREAM_LIVE_STREAM_WITHOUT_EXTENSION), xtream_live_stream_without_extension, ConfigInputOptionsFormAction::XtreamLiveStreamWithoutExtension) }
</div>
</TitledCard>
<TitledCard title={translate.t(LABEL_RESOLVE)}>
<div class="tp__config-view__cols-3">
{ edit_field_bool!(input_options_state, translate.t(LABEL_XTREAM_SKIP_VOD), resolve_vod, ConfigInputOptionsFormAction::ResolveVod) }
{ edit_field_bool!(input_options_state, translate.t(LABEL_XTREAM_SKIP_SERIES), resolve_series, ConfigInputOptionsFormAction::ResolveSeries) }
</div>
<div class="tp__config-view__cols-2">
{ edit_field_number_u16!(input_options_state, translate.t(LABEL_RESOLVE_DELAY_SEC), resolve_delay, ConfigInputOptionsFormAction::ResolveDelay) }
</div>
{ edit_field_bool!(input_options_state, translate.t(LABEL_RESOLVE_BACKGROUND), resolve_background, ConfigInputOptionsFormAction::ResolveBackground) }
</TitledCard>
<TitledCard title={translate.t(LABEL_PROBE)}>
<div class="tp__config-view__cols-3">
{ edit_field_bool!(input_options_state, translate.t(LABEL_XTREAM_SKIP_LIVE), probe_live, ConfigInputOptionsFormAction::ProbeLive) }
{ edit_field_bool!(input_options_state, translate.t(LABEL_XTREAM_SKIP_VOD), probe_vod, ConfigInputOptionsFormAction::ProbeVod) }
{ edit_field_bool!(input_options_state, translate.t(LABEL_XTREAM_SKIP_SERIES), probe_series, ConfigInputOptionsFormAction::ProbeSeries) }
</div>
<div class="tp__config-view__cols-2">
{ edit_field_number_u32!(input_options_state, translate.t(LABEL_PROBE_LIVE_INTERVAL_HOURS), probe_live_interval_hours, ConfigInputOptionsFormAction::ProbeLiveIntervalHours) }
</div>
</TitledCard>
</>
})
}
})}
<TitledCard title={translate.t(LABEL_METADATA)}>
{ edit_field_bool!(input_options_state, translate.t(LABEL_RESOLVE_TMDB), resolve_tmdb, ConfigInputOptionsFormAction::ResolveTmdb) }
{ edit_field_bool!(input_options_state, translate.t(LABEL_PROBE_STREAM), probe_stream, ConfigInputOptionsFormAction::ProbeStream) }
</TitledCard>
</TitledCard>
</Card>
}
};
frontend/src/app/components/source_editor/output_xtream_form.rs
+1 -35
View File
@@ -3,8 +3,7 @@ use crate::{
config::HasFormData, BlockId, BlockInstance, Card, EditMode, FilterInput, IconButton, Panel,
SourceEditorContext, TextButton, TitledCard, TraktListItemForm,
},
config_field_child, edit_field_bool, edit_field_number_u16, edit_field_number_u32, edit_field_text,
generate_form_reducer,
config_field_child, edit_field_bool, edit_field_text, generate_form_reducer,
};
use shared::{
error::TuliproxError,
@@ -25,11 +24,6 @@ const LABEL_SKIP_DIRECT_SOURCE: &str = "LABEL.SKIP_DIRECT_SOURCE";
const LABEL_LIVE: &str = "LABEL.LIVE";
const LABEL_VOD: &str = "LABEL.VOD";
const LABEL_SERIES: &str = "LABEL.SERIES";
const LABEL_RESOLVE: &str = "LABEL.RESOLVE";
const LABEL_PROBE: &str = "LABEL.PROBE";
const LABEL_RESOLVE_DELAY_SEC: &str = "LABEL.RESOLVE_DELAY_SEC";
const LABEL_RESOLVE_BACKGROUND: &str = "LABEL.RESOLVE_BACKGROUND";
const LABEL_PROBE_LIVE_INTERVAL_HOURS: &str = "LABEL.PROBE_LIVE_INTERVAL_HOURS";
const LABEL_FILTER: &str = "LABEL.FILTER";
const LABEL_TRAKT_API_KEY: &str = "LABEL.API_KEY";
const LABEL_TRAKT_API_VERSION: &str = "LABEL.API_VERSION";
@@ -104,14 +98,6 @@ generate_form_reducer!(
SkipLiveDirectSource => skip_live_direct_source: bool,
SkipVideoDirectSource => skip_video_direct_source: bool,
SkipSeriesDirectSource => skip_series_direct_source: bool,
ResolveSeries => resolve_series: bool,
ResolveVod => resolve_vod: bool,
ProbeSeries => probe_series: bool,
ProbeVod => probe_vod: bool,
ProbeLive => probe_live: bool,
ResolveDelay => resolve_delay: u16,
ResolveBackground => resolve_background: bool,
ProbeLiveIntervalHours => probe_live_interval_hours: u32,
Filter => filter: Option<String>,
}
);
@@ -235,26 +221,6 @@ pub fn XtreamTargetOutputView(props: &XtreamTargetOutputViewProps) -> Html {
{ edit_field_bool!(output_form_state, translate.t(LABEL_SERIES), skip_series_direct_source, XtreamTargetOutputFormAction::SkipSeriesDirectSource) }
</div>
</TitledCard>
<TitledCard title={translate.t(LABEL_RESOLVE)}>
<div class="tp__config-view__cols-3">
{ edit_field_bool!(output_form_state, translate.t(LABEL_VOD), resolve_vod, XtreamTargetOutputFormAction::ResolveVod) }
{ edit_field_bool!(output_form_state, translate.t(LABEL_SERIES), resolve_series, XtreamTargetOutputFormAction::ResolveSeries) }
</div>
<div class="tp__config-view__cols-2">
{ edit_field_number_u16!(output_form_state, translate.t(LABEL_RESOLVE_DELAY_SEC), resolve_delay, XtreamTargetOutputFormAction::ResolveDelay) }
</div>
{ edit_field_bool!(output_form_state, translate.t(LABEL_RESOLVE_BACKGROUND), resolve_background, XtreamTargetOutputFormAction::ResolveBackground) }
</TitledCard>
<TitledCard title={translate.t(LABEL_PROBE)}>
<div class="tp__config-view__cols-3">
{ edit_field_bool!(output_form_state, translate.t(LABEL_LIVE), probe_live, XtreamTargetOutputFormAction::ProbeLive) }
{ edit_field_bool!(output_form_state, translate.t(LABEL_VOD), probe_vod, XtreamTargetOutputFormAction::ProbeVod) }
{ edit_field_bool!(output_form_state, translate.t(LABEL_SERIES), probe_series, XtreamTargetOutputFormAction::ProbeSeries) }
</div>
<div class="tp__config-view__cols-2">
{ edit_field_number_u32!(output_form_state, translate.t(LABEL_PROBE_LIVE_INTERVAL_HOURS), probe_live_interval_hours, XtreamTargetOutputFormAction::ProbeLiveIntervalHours) }
</div>
</TitledCard>
{ config_field_child!(translate.t(LABEL_FILTER), {
html! {
<FilterInput filter={output_form_state_1.form.filter.clone()} on_change={Callback::from(move |new_filter| {
shared/Cargo.toml
+3 -2
View File
@@ -20,7 +20,7 @@ base64 = "0"
blake3 = "1"
fastrand = { version = "2", features = ["js"] }
zeroize = "1"
chrono = "0.4.42"
chrono = "0.4.43"
bytes = "1"
ciborium = "0.2.2"
hex = "0.4.3"
@@ -29,7 +29,8 @@ deunicode = "1.6.2"
serde-saphyr = "0.0.14"
dashmap = "6.1.0"
serde_tuple = "0.5"
paste = "1"
[target.'cfg(target_arch = "wasm32")'.dependencies]
js-sys = "0.3.83"
js-sys = "0.3.85"
getrandom = { version = "0.3", features = ["wasm_js"] }
shared/src/model/config/input.rs
+50 -9
View File
@@ -5,8 +5,10 @@ use crate::{
info_err_res,
model::EpgConfigDto,
utils::{
arc_str_serde, arc_str_vec_serde, default_as_true, deserialize_timestamp, get_credentials_from_url_str,
get_trimmed_string, is_blank_optional_string, is_false, is_true, is_zero_u16, parse_provider_scheme_url_parts,
arc_str_serde, arc_str_vec_serde, default_as_true, default_probe_live_interval, default_resolve_background,
default_resolve_delay_secs, default_xtream_live_stream_use_prefix, deserialize_timestamp,
get_credentials_from_url_str, get_trimmed_string, is_blank_optional_string, is_default_probe_live_interval,
is_default_resolve_delay_secs, is_false, is_true, is_zero_u16, parse_provider_scheme_url_parts,
sanitize_sensitive_info, serialize_option_vec_flow_map_items, trim_last_slash, Internable,
PROVIDER_SCHEME_PREFIX,
},
@@ -169,14 +171,32 @@ pub struct ConfigInputOptionsDto {
pub xtream_skip_vod: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub xtream_skip_series: bool,
#[serde(default = "default_as_true", skip_serializing_if = "is_true")]
#[serde(default = "default_xtream_live_stream_use_prefix", skip_serializing_if = "is_true")]
pub xtream_live_stream_use_prefix: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub xtream_live_stream_without_extension: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub resolve_tmdb: bool,
#[serde(default, alias = "analyze_stream", skip_serializing_if = "is_false")]
pub probe_stream: bool,
#[serde(default = "default_resolve_background", skip_serializing_if = "is_true")]
pub resolve_background: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub resolve_series: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub resolve_vod: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub probe_series: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub probe_vod: bool,
#[serde(default = "default_resolve_delay_secs", skip_serializing_if = "is_default_resolve_delay_secs")]
pub resolve_delay: u16,
#[serde(default, alias = "resolve_live", skip_serializing_if = "is_false")]
pub probe_live: bool,
#[serde(
default = "default_probe_live_interval",
alias = "resolve_live_interval_hours",
skip_serializing_if = "is_default_probe_live_interval"
)]
pub probe_live_interval_hours: u32,
}
impl Default for ConfigInputOptionsDto {
@@ -185,10 +205,17 @@ impl Default for ConfigInputOptionsDto {
xtream_skip_live: false,
xtream_skip_vod: false,
xtream_skip_series: false,
xtream_live_stream_use_prefix: default_as_true(),
xtream_live_stream_use_prefix: default_xtream_live_stream_use_prefix(),
xtream_live_stream_without_extension: false,
resolve_tmdb: false,
probe_stream: false,
resolve_background: default_resolve_background(),
resolve_series: false,
resolve_vod: false,
probe_series: false,
probe_vod: false,
resolve_delay: default_resolve_delay_secs(),
probe_live: false,
probe_live_interval_hours: default_probe_live_interval(),
}
}
}
@@ -201,7 +228,14 @@ impl ConfigInputOptionsDto {
&& self.xtream_live_stream_use_prefix
&& !self.xtream_live_stream_without_extension
&& !self.resolve_tmdb
&& !self.probe_stream
&& self.resolve_background
&& !self.resolve_series
&& !self.resolve_vod
&& !self.probe_series
&& !self.probe_vod
&& is_default_resolve_delay_secs(&self.resolve_delay)
&& !self.probe_live
&& is_default_probe_live_interval(&self.probe_live_interval_hours)
}
pub fn clean(&mut self) {
@@ -211,7 +245,14 @@ impl ConfigInputOptionsDto {
self.xtream_live_stream_use_prefix = default_as_true();
self.xtream_live_stream_without_extension = false;
self.resolve_tmdb = false;
self.probe_stream = false;
self.resolve_background = default_as_true();
self.resolve_series = false;
self.resolve_vod = false;
self.probe_series = false;
self.probe_vod = false;
self.resolve_delay = default_resolve_delay_secs();
self.probe_live = false;
self.probe_live_interval_hours = default_probe_live_interval();
}
}
shared/src/model/config/target.rs
+3 -38
View File
@@ -7,13 +7,12 @@ use crate::{
ProcessingOrder, StrmExportStyle, TargetType, TraktConfigDto,
},
utils::{
default_as_default, default_as_true, default_probe_live_interval, default_resolve_delay_secs,
is_blank_optional_string, is_config_target_options_empty, is_default_probe_live_interval,
is_default_processing_order, is_default_resolve_delay_secs, is_false, is_true, is_zero_u16,
default_as_default, default_as_true, is_blank_optional_string, is_config_target_options_empty,
is_default_processing_order, is_false, is_true, is_zero_u16,
},
};
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, Default, PartialEq)]
#[derive(Default, Debug, Clone, serde::Serialize, serde::Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct ConfigTargetOptions {
#[serde(default, skip_serializing_if = "is_false")]
@@ -46,26 +45,6 @@ pub struct XtreamTargetOutputDto {
pub skip_video_direct_source: bool,
#[serde(default = "default_as_true", skip_serializing_if = "is_true")]
pub skip_series_direct_source: bool,
#[serde(default = "default_as_true", skip_serializing_if = "is_true")]
pub resolve_background: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub resolve_series: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub resolve_vod: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub probe_series: bool,
#[serde(default, skip_serializing_if = "is_false")]
pub probe_vod: bool,
#[serde(default = "default_resolve_delay_secs", skip_serializing_if = "is_default_resolve_delay_secs")]
pub resolve_delay: u16,
#[serde(default, alias = "resolve_live", skip_serializing_if = "is_false")]
pub probe_live: bool,
#[serde(
default = "default_probe_live_interval",
alias = "resolve_live_interval_hours",
skip_serializing_if = "is_default_probe_live_interval"
)]
pub probe_live_interval_hours: u32,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub trakt: Option<TraktConfigDto>,
#[serde(default, skip_serializing_if = "is_blank_optional_string")]
@@ -80,14 +59,6 @@ impl Default for XtreamTargetOutputDto {
skip_live_direct_source: default_as_true(),
skip_video_direct_source: default_as_true(),
skip_series_direct_source: default_as_true(),
resolve_background: default_as_true(),
resolve_series: false,
resolve_vod: false,
probe_series: false,
probe_vod: false,
resolve_delay: default_resolve_delay_secs(),
probe_live: false,
probe_live_interval_hours: default_probe_live_interval(),
trakt: None,
filter: None,
t_filter: None,
@@ -110,12 +81,6 @@ impl XtreamTargetOutputDto {
self.skip_live_direct_source
|| self.skip_video_direct_source
|| self.skip_series_direct_source
|| self.resolve_background
|| self.resolve_series
|| self.resolve_vod
|| self.probe_series
|| self.probe_vod
|| self.probe_live
|| self.trakt.is_some()
|| self.filter.is_some()
}
shared/src/model/playlist.rs
+25 -18
View File
@@ -1,4 +1,5 @@
use crate::{
create_bitset,
model::{
xtream_const, ClusterFlags, CommonPlaylistItem, ConfigTargetOptions, EpisodeStreamProperties,
SeriesStreamProperties, StreamProperties, UUIDType, VideoStreamProperties, XtreamInfoDocument,
@@ -617,12 +618,17 @@ impl From<M3uPlaylistItem> for CommonPlaylistItem {
fn from(item: M3uPlaylistItem) -> Self { item.to_common() }
}
#[allow(clippy::struct_excessive_bools)]
create_bitset!(
u8,
XtreamMappingFlags,
SkipLiveDirectSource,
SkipVideoDirectSource,
SkipSeriesDirectSource,
RewriteResourceUrl
);
pub struct XtreamMappingOptions {
pub skip_live_direct_source: bool,
pub skip_video_direct_source: bool,
pub skip_series_direct_source: bool,
pub rewrite_resource_url: bool,
pub flags: XtreamMappingFlagsSet,
pub force_redirect: Option<ClusterFlags>,
pub reverse_item_types: PlaylistItemTypeSet,
pub username: String,
@@ -641,19 +647,20 @@ impl XtreamMappingOptions {
virtual_id: VirtualId,
) -> Option<String> {
let is_reverse = self.is_reverse(item_type);
let resource_url = if is_reverse && self.rewrite_resource_url && self.base_url.is_some() {
let resource_url = format!(
"{}/resource/{}/{}/{}/{}",
self.base_url.as_ref().map_or_else(String::new, |b| b.clone()),
xtream_cluster.as_stream_type(),
self.username,
self.password,
virtual_id
);
Some(resource_url)
} else {
None
};
let resource_url =
if is_reverse && self.flags.contains(XtreamMappingFlags::RewriteResourceUrl) && self.base_url.is_some() {
let resource_url = format!(
"{}/resource/{}/{}/{}/{}",
self.base_url.as_ref().map_or_else(String::new, |b| b.clone()),
xtream_cluster.as_stream_type(),
self.username,
self.password,
virtual_id
);
Some(resource_url)
} else {
None
};
resource_url
}
}
shared/src/model/playlist_document.rs
+13 -5
View File
@@ -1,9 +1,9 @@
use crate::{
model::{
info_doc_utils::InfoDocUtils, LiveStreamProperties, SeriesStreamProperties, StreamProperties,
VideoStreamProperties, XtreamCluster, XtreamEmptyDoc, XtreamInfoDocument, XtreamMappingOptions,
XtreamPlaylistItem, XtreamSeriesInfoData, XtreamSeriesInfoDoc, XtreamVideoInfoData, XtreamVideoInfoDoc,
XtreamVideoMovieData,
VideoStreamProperties, XtreamCluster, XtreamEmptyDoc, XtreamInfoDocument, XtreamMappingFlags,
XtreamMappingOptions, XtreamPlaylistItem, XtreamSeriesInfoData, XtreamSeriesInfoDoc, XtreamVideoInfoData,
XtreamVideoInfoDoc, XtreamVideoMovieData,
},
utils::{arc_str_option_serde, arc_str_serde, arc_str_vec_serde, Internable},
};
@@ -273,7 +273,11 @@ impl XtreamPlaylistItem {
category_ids: vec![self.category_id],
container_extension: video.container_extension.clone(),
custom_sid: video.custom_sid.clone(),
direct_source: if options.skip_video_direct_source { empty_str } else { video.direct_source.clone() },
direct_source: if options.flags.contains(XtreamMappingFlags::SkipVideoDirectSource) {
empty_str
} else {
video.direct_source.clone()
},
})
}
@@ -294,7 +298,11 @@ impl XtreamPlaylistItem {
category_ids: vec![self.category_id],
custom_sid: live.custom_sid.clone(),
tv_archive: live.tv_archive.unwrap_or_default(),
direct_source: if options.skip_live_direct_source { empty_str } else { live.direct_source.clone() },
direct_source: if options.flags.contains(XtreamMappingFlags::SkipLiveDirectSource) {
empty_str
} else {
live.direct_source.clone()
},
tv_archive_duration: live.tv_archive_duration.unwrap_or_default(),
})
}
shared/src/model/playlist_info_document.rs
+3 -3
View File
@@ -3,7 +3,7 @@ use crate::{
model::{
info_doc_utils::InfoDocUtils, PlaylistItemType, SeriesStreamDetailEpisodeProperties,
SeriesStreamDetailSeasonProperties, SeriesStreamProperties, StreamProperties, VideoStreamProperties, VirtualId,
XtreamCluster, XtreamMappingOptions,
XtreamCluster, XtreamMappingFlags, XtreamMappingOptions,
},
utils::{arc_str_option_serde, arc_str_serde, arc_str_vec_serde, Internable},
};
@@ -423,7 +423,7 @@ impl StreamProperties {
category_ids: vec![category_id],
container_extension: Arc::clone(&video.container_extension),
custom_sid: video.custom_sid.as_ref().map(Arc::clone),
direct_source: if options.skip_video_direct_source {
direct_source: if options.flags.contains(XtreamMappingFlags::SkipVideoDirectSource) {
Arc::clone(&empty_str)
} else {
Arc::clone(&video.direct_source)
@@ -516,7 +516,7 @@ impl StreamProperties {
custom_sid: ep.custom_sid.as_ref().map(Arc::clone),
added: Arc::clone(&ep.added),
season: ep.season,
direct_source: if options.skip_series_direct_source {
direct_source: if options.flags.contains(XtreamMappingFlags::SkipSeriesDirectSource) {
Arc::clone(&empty_str)
} else {
Arc::clone(&ep.direct_source)
shared/src/utils/bitset.rs
+15 -4
View File
@@ -1,5 +1,5 @@
#[macro_export]
macro_rules! create_bit_set {
macro_rules! create_bitset {
($storage:ty, $enum_name:ident, $($variant:ident),*) => {
paste::paste! {
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, serde::Serialize, serde::Deserialize)]
@@ -34,19 +34,19 @@ macro_rules! create_bit_set {
pub fn from_variants(variants: &[$enum_name]) -> Self {
let mut set = Self::new();
for &v in variants {
set.add(v);
set.set(v);
}
set
}
#[inline(always)]
pub fn add(&mut self, variant: $enum_name) {
pub fn set(&mut self, variant: $enum_name) {
// LLVM can optimize this to a single 'bts' or 'or' instruction
self.0 |= 1 << (variant as $storage);
}
#[inline(always)]
pub fn remove(&mut self, variant: $enum_name) {
pub fn unset(&mut self, variant: $enum_name) {
self.0 &= !(1 << (variant as $storage));
}
@@ -121,3 +121,14 @@ macro_rules! create_bit_set {
}
};
}
#[macro_export]
macro_rules! apply_flags {
($dto:expr, $set:expr, $enum_type:path; $(($field:ident, $variant:ident)),* $(,)?) => {
$(
if $dto.$field {
$set.set(<$enum_type>::$variant);
}
)*
};
}
shared/src/utils/default_utils.rs
+3
View File
@@ -116,6 +116,9 @@ pub const fn default_probe_live_interval() -> u32 { 120 }
pub const fn is_default_probe_live_interval(v: &u32) -> bool { *v == default_probe_live_interval() }
pub const fn default_resolve_background() -> bool { true }
pub const fn default_xtream_live_stream_use_prefix() -> bool { true }
//////////////////////////////
// HDHomerun Device Defaults
//////////////////////////////