mirror of
https://github.com/vinefeeder/TwinVine.git
synced 2026-07-15 18:10:04 +02:00
initial commit
This commit is contained in:
VineFeeder
@@ -0,0 +1,419 @@
|
||||
# VineFeeder
|
||||
|
||||
VineFeeder is a dynamic Python-based application and **framework** that presents a graphical user interface (GUI) to act as a front-end for Devine (https://github.com/devine-dl/devine) or envied (https://github.com/envied-dl/envied) - each a video-downloader, envied being a fork of devine.
|
||||
|
||||
Vinefeeder enables users to more easily interact with Devine's streaming services. The project allows users to browse and select media content from multiple services, each of which is dynamically loaded as a separate module. Each service has a configuration which will work out of the box, but Devine download options may be set on a service by service basis.
|
||||
|
||||
Features:
|
||||
|
||||
*Dynamic Service Loading*: Services are loaded dynamically from a predefined folder.
|
||||
Each service has its own configuration and logic, allowing flexible media handling.
|
||||
|
||||
*Service-Specific Parsing*: Each service can define its own method of parsing media
|
||||
content, making the platform adaptable to different service architectures.
|
||||
Additionally each service may store a string of download options for devine such as
|
||||
video resolution or subtitle format.
|
||||
|
||||
*Alphabetical Service Listing*: Services are displayed as clickable buttons in
|
||||
alphabetical order, enhancing the user experience.
|
||||
|
||||
*GUI Interface*: The interface is built using PyQt6, offering a user-friendly experience
|
||||
for selecting media content.
|
||||
|
||||
*Category Browsing*: Services support browsing by category, allowing users to explore
|
||||
content within specific genres or collections.
|
||||
|
||||
*Video Download*: Once media content is selected, videos can be downloaded directly
|
||||
from the service using predefined tools (e.g., devine).
|
||||
|
||||
*Ultra-High-definition*: Videos in UHD are automatically requested from sites that provide such quality.
|
||||
hlg_status in config.yaml may be set to False to not do this.
|
||||
|
||||
*Precision Episode Selection*: Vinefeeder handles the selection of episodes and 'feeds'
|
||||
Devine one-by-one, making Devine's -w switch defunct.
|
||||
|
||||
|
||||
**Installation**
|
||||
|
||||
Requirements
|
||||
**python 3.12**
|
||||
|
||||
Python 3.12 or higher Note: PyQT6 needs PyQt6.QtWidgets
|
||||
Required Python modules (install via pip):
|
||||
beaupy
|
||||
devine
|
||||
httpx
|
||||
jmespath
|
||||
PyQt6
|
||||
PyQt6_sip
|
||||
PyYAML
|
||||
rich
|
||||
Scrapy
|
||||
|
||||
You can install the necessary VineFeeder packages with bash or a Window's Terminal. Make sure to install python modules to the same python environment (env) as Devine
|
||||
First be sure to follow Devine or envied's install and set-up procedure and ensure it all works correctly to your liking.
|
||||
|
||||
**If you have installed Devine using Poetry and you run Devine by 'poetry run devine' then install devine again using pip, keeping the poetry install for when you use the command line directly. The pip installed version is the developer's recommended way. If you go off-piste you may break something! Both versions sit happily together**
|
||||
|
||||
**If you have installed Devine by any other way than via poetry or by 'pip install devine' then remove it and re-install using the correct method before running VineFeeder! The re-install should pick-up the last configuration. if not re-configure Devine. Make absolutely sure devine can be called from any folder on your system.**
|
||||
|
||||
**If you are using envied make sure you have installed the tool version that runs from the command envied. See https://github.com/envied-dl/envied for installation details**
|
||||
|
||||
**Setup**
|
||||
|
||||
if you've installed Devine or envied , via pip or uv, in a specific python environment (virtual environment - venv) , first start that environment before installing VineFeeder to the same environment.
|
||||
|
||||
For all users, after devine/envied is configured
|
||||
|
||||
Clone the repository:
|
||||
With bash or a Window's Terminal
|
||||
|
||||
git clone https://github.com/vinefeeder/VineFeeder.git
|
||||
cd VineFeeder
|
||||
pip install -r requirements.txt
|
||||
|
||||
will install a VineFeeder folder with all files. Change directory, move into VineFeeder
|
||||
|
||||
|
||||
**Usage**
|
||||
|
||||
From the VineFeeder folder run the application by executing the main script:
|
||||
|
||||
With bash or a Window's Terminal
|
||||
|
||||
python vinefeeder.py
|
||||
or
|
||||
python vinefeeder.py -help
|
||||
|
||||
As set-up on start it will run with zero modification if Devine is correctly installed. There are six UK services and TVNZ active.
|
||||
Regard TVNZ as experimental for the time being. For one thing Devine's TVNZ service fails on some
|
||||
Sports titles and the VineFeeder service is still in testing. If Devine fails use the URL with Ozivine -
|
||||
there are reports it works. This Ozivine fork has a downloader https://github.com/liam8888999/ozivine
|
||||
|
||||
It is recommended to use PowerShell or Terminal in Windows and a Unix-like terminal in Linux - but select, via
|
||||
preferences, a dark background for better contrast with the colours used in Vinefeeder.
|
||||
|
||||
Note: envied has an issue releasing screen control back to VineFeeder when a download is finished. There is an option in the top-level config.yaml to reset the terminal after a download. As delivered the tetrminal-reset option is off. I don't think the issue appears with Windows; it has not been tested.
|
||||
The reset clears screen history - which you may not want. So use batch mode. Select as many downloads as you need and the run the batch download. Download history will remain on the screen until all dowloads are finished.
|
||||
|
||||
**Interacting with Services**
|
||||
|
||||
Once the GUI is launched, you may interact with various streaming services by clicking on their corresponding buttons.
|
||||
The 'URL or search' box MAY be used for an immediate search entry or direct-download URL or it may be left empty.
|
||||
If left empty a menu is offered.
|
||||
A search for a series-name will usually end with a list of episodes to select from.
|
||||
If a series-url is pasted in the text-box then Vinefeeder will immediatly pass the url to the downloader. And being a series-url the downloader will download the complete series. So if you wish to select episodes use a search term.
|
||||
|
||||
**Help**
|
||||
|
||||
Starting VineFeeder with python vinefeeder.py --help will show options to set a service congfiguration - if required.
|
||||
In the top level folder is config.yaml. Open the file in a text editor and set your preferred DOWNLOAD_ORCHESTRATOR to devine or envied.
|
||||
|
||||
**Service Configuration**
|
||||
|
||||
Most users will not need alter any configuration to download videos!
|
||||
|
||||
However, if you have, in your past use of Devine resorted to setting Devine-download options other than -w etc.
|
||||
You may continue to do with VineFeeder only you do so just once, in the config.yaml for each service.
|
||||
Each service's config.yaml has an 'options:' entry just add the string you would use with devine e.g -q 720 to
|
||||
ensure only 720p resolution videos are selected.
|
||||
|
||||
To open a config.yaml for a service:-
|
||||
|
||||
python vinefeeder.py --service-folder <SERVICE-NAME>
|
||||
python vinefeeder.py --service-folder ALL4, for example
|
||||
|
||||
Edit the line starting 'options'. Use exactly the same syntax as Devine would require on its command line
|
||||
|
||||
Image
|
||||
|
||||
|
||||
|
||||
Services are displayed in alphabetical order for easy access. From there, you can:
|
||||
|
||||
Browse Categories: Select a media category to view available content.
|
||||
Search by URL: Input a direct URL to download media content.
|
||||
Download Media: Select a media item to start the download process.
|
||||
|
||||
**Services**
|
||||
|
||||
Currently seven services are working - All4, BBC, ITVX, My5 STV, U and TPTV , all UK services plus TVNZ.
|
||||
Other services: awaiting contributors!!
|
||||
|
||||
**Text Entry Box**
|
||||
|
||||
This will be the predominant method of using VineFeeder. Search on a keyword to locate a
|
||||
programme series and click on its title when search finishes, to select any combination of
|
||||
episodes for download.
|
||||
|
||||
Additionlly the search box will take a URL for direct download. The URL may be for a single
|
||||
episode and it will download directly. If you feed a URL for a *series of episode* i.e a
|
||||
series-link then Devine will download the series.
|
||||
|
||||
The text-entry-box is cleared automatically after use. Once a download successfully finishes
|
||||
the screen displays 'Ready!' and waits for further entry.
|
||||
|
||||
**Series numbers**
|
||||
|
||||
Normally services will provide enough data to collect a series number.
|
||||
When series numbers are not available, Vinfeeder sets a default number of 100.
|
||||
This is for housekeeping reasons. Devine labelling defaults to 00 for any saved
|
||||
videos without a specified series number.
|
||||
|
||||
**Backing out**
|
||||
|
||||
If you have followed a path to a selection and do not wish to continue then,
|
||||
in most circumstances, depending on operating system, ctrl-C will return you
|
||||
to a 'Ready!' status display on the screen within a second or two, and await
|
||||
further operations.
|
||||
|
||||
**Batch Mode**
|
||||
|
||||
To select Batch Mode, move the GUI's batch-mode slider to the right.
|
||||
When batch-mode is selected all devine download commands will be saved to
|
||||
a file, batch.txt, created in the top level folder of VineFeeder.
|
||||
|
||||
*BE SURE TO SELECT BATCH MODE BEFORE SELECTING A SERVICE!*
|
||||
|
||||
The batch file will only be created once a service's video download is initiated.
|
||||
|
||||
A banner in the GUI will then display the existence of batch.txt.
|
||||
Any service may be used to add to batch.txt for later batch download.
|
||||
|
||||
Clicking the Run Batch button will then download all the selected videos.
|
||||
|
||||
After devine's downloads are complete, the option to delete the batch file will
|
||||
appear. The file may also be manually deleted.
|
||||
|
||||
Note: While batch-run is downloading you may still add videos to batch.txt if you paste
|
||||
the video link for a service into VineFeeder's text box. You may not interact with a service
|
||||
if the service needs to use Terminal to display.
|
||||
|
||||
Batch mode preserves state; you may close VineFeeder down, if you wish,
|
||||
and resume later.
|
||||
|
||||
**UHD**
|
||||
|
||||
Currently only the BBC service carries HLG output. Vinefeeder will attempt to automatically
|
||||
detect if HLG videos are available. Software compares the search term to a fetched list of
|
||||
HLG titles. For this reason the search term *must* accurately match the programme title.
|
||||
The BBC programme "The Gold" will only be found with search terms 'the gold'; 'The Gold'
|
||||
or upper/lower-case variations.
|
||||
The current BBC H.265 output:
|
||||
|
||||
1. A Good Girl's Guide to Murder
|
||||
2. A Perfect Planet
|
||||
3. Asia
|
||||
4. Attenborough and the Giant Sea Monster
|
||||
5. Attenborough's Wonder of Song
|
||||
6. Blue Lights
|
||||
7. Blue Planet II
|
||||
8. Boarders
|
||||
9. Boat Story
|
||||
10. Boiling Point
|
||||
11. Champion
|
||||
12. Cunk on Life
|
||||
13. Doctor Who
|
||||
14. Doctor Who: 60th Anniversary Special
|
||||
15. Doctor Who: Revolution of the Daleks
|
||||
16. Doctor Who: The Power of the Doctor
|
||||
17. Domino Day
|
||||
18. Dynasties
|
||||
19. Everything I know about Love
|
||||
20. Families Like Ours
|
||||
21. Frozen Planet II
|
||||
22. Fungi: The Web of Life
|
||||
23. Glastonbury
|
||||
24. His Dark Materials
|
||||
25. Industry, Series 2 and 3
|
||||
26. Life After Life
|
||||
27. Mammals
|
||||
28. Men Up
|
||||
29. Mood
|
||||
30. Mr Loverman
|
||||
31. Murder is Easy
|
||||
32. Nightsleeper
|
||||
33. Peaky Blinders, Series 6
|
||||
34. Planet Earth III
|
||||
35. Red Rose
|
||||
36. Reunion
|
||||
37. SAS Rogue Heroes
|
||||
38. Serengeti, Series 3
|
||||
39. Seven Worlds, One Planet
|
||||
40. Sherwood
|
||||
41. Showtrial
|
||||
42. Squad Goals: Dorking 'Til I Die
|
||||
43. Storyville: Your Fat Friend
|
||||
44. Strike: The Ink Black Heart
|
||||
45. Swan Lake from English National Ballet
|
||||
46. Ten Pound Poms
|
||||
47. The Americas
|
||||
48. The Bombing of Pan Am 103
|
||||
49. The Boy, the Mole, the Fox and the Horse
|
||||
50. The English
|
||||
51. The Gold
|
||||
52. The Green Planet
|
||||
53. The Jetty
|
||||
54. The Mating Game
|
||||
55. The Reckoning
|
||||
56. The Responder
|
||||
57. The Sixth Commandment
|
||||
58. The Tourist
|
||||
59. The Way
|
||||
60. The Woman in the Wall
|
||||
61. This City is Ours
|
||||
62. This Town
|
||||
63. Towards Zero
|
||||
64. Vigil
|
||||
65. What It Feels Like for a Girl
|
||||
66. Wild Isles
|
||||
67. Wreck
|
||||
| |
|
||||
#### NOTE
|
||||
In the series above not every episode is available in HLG. Glastonbury is particularly
|
||||
tricky. When Devine is told to expect HLG and then finds no output, devine will throw an error.
|
||||
E.g this fails; devine dl -r HLG --list iP https://www.bbc.co.uk/iplayer/episodes/b007r6vx/glastonbury
|
||||
as some output is SDR
|
||||
A way around this is to start the service editor with the command: " python vinefeeder.py --service-folder BBC".
|
||||
Then edit the line hlg_status: True; change True to False, and save the file. This turns off HLG from being downloaded.
|
||||
|
||||
**Closing Down**
|
||||
|
||||
The GUI should first be shutdown by mouse-clicking the top right X. The GUI then should
|
||||
release control of the Terminal. ctrl-C in the Terminal may be required if shutting down
|
||||
following an error.
|
||||
|
||||
**Custom Services**
|
||||
|
||||
Vinefeeder is a *framework* too. It has been written in such a way that most features
|
||||
needed to scape and parse a website are already written and available.
|
||||
|
||||
If you feel moderately confident with python, then writing a new service to allow Devine
|
||||
to be run more interactively, is relatively straightforward. Most of the processes for
|
||||
downloading, parsing and displaying for selection form part of the base functions and
|
||||
are already written in this framework.
|
||||
|
||||
Any new service will need an __init__.py to be written and to implement just four
|
||||
methods.
|
||||
|
||||
The ultimate aim of a services' __init__.py is to extract a list of video-data from a content
|
||||
provide and store it a numerous dict items in a list container. The video-data is collected in
|
||||
dict objects labelled 'episode'.
|
||||
|
||||
Each episode has
|
||||
|
||||
episode = {
|
||||
'series_no': ser_number,
|
||||
'title': ep_number,
|
||||
'url': url,
|
||||
'synopsis': synopsis
|
||||
}
|
||||
|
||||
Note 'title' may be an episode number or descriptive text. It varies with provider
|
||||
The dict is added to the Vinefeeder storage list by:-
|
||||
|
||||
self.add_episode(brand_slug, episode)
|
||||
|
||||
Note brand_slug is a series-name. self.add_episode creates a lists of episode dicts
|
||||
This happens in the __init__.py super-class of BaseLoader
|
||||
|
||||
|
||||
To add a new service:
|
||||
|
||||
Create a new folder for the service inside the services directory.
|
||||
Add the following:
|
||||
A config.yaml file with the media_dict. see examples in the existing services folder.
|
||||
|
||||
An __init__.py file defining the loader class for the service.
|
||||
The name of the loader class is constrained. It must take the service tag name, e.g. TVNZ
|
||||
and use that to create the class-name TvnzLoader. This allows vinefeeder.py discover and
|
||||
to call its use later.
|
||||
Note: the loader class file MUST inherit (or sub-class) BaseLoader see ALL4/__init__.py
|
||||
as an example.
|
||||
Note: Most web-sites that provide on-demand streaming have a 'browse' or 'category'
|
||||
page where video categories may be selected for view. Use some/all of these links to
|
||||
produce a media_dict of ( catergory: link, }
|
||||
|
||||
There are 4 methods to implement
|
||||
receive
|
||||
fetch_videos
|
||||
second_fetch
|
||||
fetch_videos_by_category
|
||||
Follow any channel as a model except BBC and U. Most sites provide json to describe their video content.
|
||||
Usually it is within a script.
|
||||
There are two methods to extract the json in parsing_utils depending on whether a script id
|
||||
is used or not.
|
||||
Parsing utils uses XPATH as a locator syntax.
|
||||
ChatGPT will helpt to find the XPATH syntax if you give it the web page and tell it
|
||||
which javascript you need.XPATH is considerably faster than alternates such as bs4.
|
||||
The BaseLoader class which your service must inherit, has methods to GET, POST or
|
||||
return OPTIONS from the web.
|
||||
DO NOT USE OTHER METHODS THAN THESE PROVIDED.
|
||||
|
||||
RECEIVE
|
||||
I find it easier to start off implementing a keyword search from the GUI text box.
|
||||
Very few changes to an existing service receive method will be needed.
|
||||
Copy one and adjust. Note the use of inx (index) to help specify action.
|
||||
Vinefeeder's GUI calls the service, passing parameters in the process such as a
|
||||
service's Devine download options.
|
||||
By definiton some receive() parameters may be empty.
|
||||
|
||||
FETCH_VIDEOS
|
||||
Fetch_videos is very service specific and will need to send data to a web-site and processs the response to provide
|
||||
an 'episode' list for display.
|
||||
Here 'episode' is a container for a web site's response of suggestion of a single video or series that match the
|
||||
search-word.
|
||||
In the example below the parsed_data contains json from which we extract to add to BaseLoader's episode list via
|
||||
self.add_episode(series_name, episode)
|
||||
|
||||
if parsed_data and 'results' in parsed_data:
|
||||
for item in parsed_data['results']:
|
||||
series_name = item.get('brand', {}).get('websafeTitle', 'Unknown Series')
|
||||
episode = {
|
||||
'title': item.get('brand', {}).get('title', 'Unknown Title'),
|
||||
'url': f"{item.get('brand', {}).get('href', '')}",
|
||||
'synopsis': item.get('brand', {}).get('description', 'No synopsis available.')
|
||||
}
|
||||
self.add_episode(series_name, episode)
|
||||
|
||||
SECOND_FETCH
|
||||
Again the process is much the same as in fetch_videos() a web site has its html harvested,
|
||||
a script is extracted from which json is pulled using the facility methods in BaseLoader and parsing _utils.
|
||||
|
||||
FETCH_VIDEO_BY_CATEGORY
|
||||
Displays the media_dict from congig.yaml - so put any category heading and links in the new service config.
|
||||
Once a category is selected, again parse the json and follow an existing service for a model answer, adjusting to suit the syntax required by our new service.
|
||||
Potentially, you may not implement this if the site does not provide a worthwhile browse by category list,
|
||||
or the data cannot be parsed readily. The service U falls into this category.
|
||||
|
||||
VineFeeder will dynamically detect and load the new services on the next start.
|
||||
|
||||
**Special Devine Commands**
|
||||
|
||||
If your use requirements are such that each video you download using Devine needs many different parameters to be passed
|
||||
to the downloader, and set different each time, this tool may not meet your needs:
|
||||
see https://github.com/billybanana80/DevineGUI
|
||||
|
||||
If however you are happy to set your download parameters once, variable for each service, the service's config.yaml,
|
||||
then this project may work well for you, and provide the front-end Devine has always lacked.
|
||||
|
||||
**Contributing**
|
||||
|
||||
Contributors are welcomed to add services to front-end those services provided by Devine - see above.
|
||||
Presently six UK services and New Zealand's TVNZ are fully configured; contributions for other services welcome.
|
||||
.
|
||||
|
||||
Images
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
**License**
|
||||
|
||||
This project is licensed under the MIT License. See the LICENSE file for more details.
|
||||
Reference in New Issue
Block a user