T

monosansandGitHub e4047e16ae Huge update

- Drop support for Python 3.7.
- Use offline geolocation database instead of ip-api.com.
- Add support for proxy authentication.
- Add pre-compiled binaries.
- Refactor the code by a lot.
- Simplify config.
- Add automatic text encoding detection.
- Use TOML instead of INI for configuration.
- Add support for using httpbin-compatible services and services which return plain IP address for getting proxy's exit node.
- Add support for reading proxy lists from local files.
- Add support for saving to json.
- Remove support for saving geolocation to txt, use json instead.
- Improve parser.
- Improve config validation.
- Use poetry instead of requirements.txt.
- Use venv in start scripts.

2024-01-21 17:45:01 +00:00

.github

Huge update

2024-01-21 17:45:01 +00:00

proxy_scraper_checker

Huge update

2024-01-21 17:45:01 +00:00

.gitignore

Huge update

2024-01-21 17:45:01 +00:00

.pre-commit-config.yaml

Huge update

2024-01-21 17:45:01 +00:00

config.toml

Huge update

2024-01-21 17:45:01 +00:00

install-termux.sh

Huge update

2024-01-21 17:45:01 +00:00

LICENSE

Many changes. See CHANGELOG

2022-02-13 00:33:49 +03:00

poetry.lock

Huge update

2024-01-21 17:45:01 +00:00

pyproject.toml

Huge update

2024-01-21 17:45:01 +00:00

README.md

Huge update

2024-01-21 17:45:01 +00:00

screenshot.png

Refactor & improve output

2023-01-02 00:09:55 +03:00

start-termux.sh

Huge update

2024-01-21 17:45:01 +00:00

start.cmd

Huge update

2024-01-21 17:45:01 +00:00

start.sh

Huge update

2024-01-21 17:45:01 +00:00

README.md

proxy-scraper-checker

HTTP, SOCKS4, SOCKS5 proxies scraper and checker.

Can determine if the proxy is anonymous.
Supports determining the geolocation of the proxy exit node.
Can sort proxies by speed.
Uses regex to find proxies of format protocol://username:password@ip:port on a web page or in a local file, allowing proxies to be extracted even from json without code changes.
Supports proxies with authentication.
It is possible to specify the URL to which to send a request to check the proxy.
Supports saving to plain text and json.
Asynchronous.

You can get proxies obtained using this project in monosans/proxy-list.

Installation and usage

Pre-compiled binary

This is the easiest way, but it is only available for x64 Windows, macOS and Linux. Just download the archive for your OS from https://nightly.link/monosans/proxy-scraper-checker/workflows/ci/main?preview, unzip it, edit config.toml and run the executable.

If Windows Defender detects an executable file as a virus, please read this.

Running from source code

Desktop

Install Python. The minimum version required is 3.8. The recommended version is 3.11, because 3.12 may not install some libraries in the absence of a C compiler.
Download and unpack the archive with the program.
Edit config.toml to your preference.
Run the script that installs dependencies and starts proxy-scraper-checker:
- On Windows run start.cmd
- On Unix-like operating systems run start.sh

Termux

To use proxy-scraper-checker in Termux, knowledge of the Unix command-line interface is required.

Download Termux from F-Droid. Don't download it from Google Play.
Run the following command (it will automatically update Termux packages, install Python, and download and install proxy-scraper-checker):
```
bash <(curl -fsSL 'https://raw.githubusercontent.com/monosans/proxy-scraper-checker/main/install-termux.sh')
```
Edit ~/proxy-scraper-checker/config.toml to your preference using a text editor (vim/nano).

To run proxy-scraper-checker use the following command:

cd ~/proxy-scraper-checker && sh start-termux.sh

Something else?

All other info is available in config.toml file.

License

MIT

This product includes GeoLite2 Data created by MaxMind, available from https://www.maxmind.com.