haeaeae/proxy-scraper-checker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
297 Commits 1 Branch 0 Tags
dca437feae0fea2ac3ade3f2b06520cf24c0bccb
Commit Graph
297 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
monosans dca437feae Add raise_for_status=True when scraping sources 2024-01-23 09:47:43 +03:00
monosans 73303cab1b Improve response decoding performance 2024-01-23 09:37:29 +03:00
monosans 31a50964eb Fix duplicates 2024-01-23 08:09:08 +03:00
monosans 0c0b23a161 Use lto=auto in compilation 2024-01-22 12:13:26 +03:00
monosans 2d6296b577 Enable full extras when compiling 2024-01-22 12:13:18 +03:00
monosans a42448808e Refactor get_grouped 2024-01-21 23:40:05 +03:00
monosans 0984fd1a82 Fix an edge case 2024-01-21 23:37:46 +03:00
monosans d42f09aeba Rename output path 2024-01-21 21:51:31 +03:00
monosans f4af55d37a Fix encoding issues 2024-01-21 21:46:01 +03:00
monosans 24f5eb5f2f Less strict parser 2024-01-21 21:24:09 +03:00
monosans 58935cd328 Remove dead sources 2024-01-21 21:19:33 +03:00
monosans 40b1d1596e Edit README.md 2024-01-21 21:00:43 +03:00
monosansandGitHub e4047e16ae Huge update 
- Drop support for Python 3.7.
- Use offline geolocation database instead of ip-api.com.
- Add support for proxy authentication.
- Add pre-compiled binaries.
- Refactor the code by a lot.
- Simplify config.
- Add automatic text encoding detection.
- Use TOML instead of INI for configuration.
- Add support for using httpbin-compatible services and services which return plain IP address for getting proxy's exit node.
- Add support for reading proxy lists from local files.
- Add support for saving to json.
- Remove support for saving geolocation to txt, use json instead.
- Improve parser.
- Improve config validation.
- Use poetry instead of requirements.txt.
- Use venv in start scripts.
 2024-01-21 17:45:01 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		2415621157 Update .pre-commit-config.yaml (#126) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-20 03:44:32 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		b0e690a466 Update .pre-commit-config.yaml (#124) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-18 03:39:57 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		4f70f33748 Update .pre-commit-config.yaml (#123) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-16 03:41:57 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		64f9a0bc94 Update .pre-commit-config.yaml (#122) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-13 03:40:00 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		3fe4deac44 Update .pre-commit-config.yaml (#121) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-12 03:39:16 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		15311ad964 Update .pre-commit-config.yaml (#120) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-03 03:42:00 +00:00
monosans eeca39b8f5 Update User-Agent 2023-12-26 10:09:22 +03:00
monosans 4211933cff Improve traceback 2023-12-25 15:11:23 +03:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		89b1cbe7bc Update .pre-commit-config.yaml (#116) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-23 03:39:42 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		914ae5ee01 Update .pre-commit-config.yaml (#115) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-22 03:41:07 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		640cb5cf6a Update .pre-commit-config.yaml (#114) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-14 03:39:06 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		cea4dffd16 Update .pre-commit-config.yaml (#113) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-11 03:40:25 +00:00
monosans 906bb8387b Reconfigure dependabot 2023-12-10 11:04:27 +03:00
monosans b9a015974e Reconfigure dependency updates 2023-12-10 10:32:29 +03:00
monosans d8179c909d Add instructions for checking local proxy lists 
Closes #22
 2023-12-09 15:36:27 +03:00
monosans 17b680d498 Update sources 2023-12-09 15:27:19 +03:00
monosans cbf10a37d3 Don't set geolocation if geolocation folders are disabled 2023-12-09 15:16:44 +03:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		d45db44c6b Update .pre-commit-config.yaml (#111) 
* Update .pre-commit-config.yaml

* Ignore PLR0917 in ruff

* Reformat

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-07 08:08:12 +00:00
monosans 58292983c8 Change Prettier pre-commit hook 2023-12-02 10:16:20 +03:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		5d972ac26d Update .pre-commit-config.yaml (#108) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-11-25 06:09:50 +00:00
monosans fb59bdaf70 Update User-Agent 2023-11-24 19:54:03 +03:00
monosans b571113fd7 Show diff on pre-commit failure 2023-11-21 21:40:41 +03:00
monosans c4116cf236 Simplify aiohttp version specifier 2023-11-20 02:14:46 +03:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		aae6e8dfbf Update .pre-commit-config.yaml (#107) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-11-18 06:09:55 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		73aef3d0fe Update .pre-commit-config.yaml (#106) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-11-14 06:11:10 +00:00
monosans ef789badb5 Fix installation on Python 3.12 in Termux 2023-11-14 01:00:13 +03:00
monosans bcf930052b Fix pyproject.toml 2023-11-14 00:58:39 +03:00
monosans f1c292f52c Fix installation on Python 3.12 2023-11-14 00:57:15 +03:00
monosans e4f05243c6 Reconfigure mypy 2023-11-14 00:44:05 +03:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		fa88a3f94e Update .pre-commit-config.yaml (#105) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-11-11 10:51:13 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		914d73be67 Update .pre-commit-config.yaml (#104) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-11-09 06:10:29 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		57696e62e3 Update .pre-commit-config.yaml (#101) 
* Update .pre-commit-config.yaml

* Fix

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-11-04 09:29:21 +00:00
monosans 813d320c64 Use logging.Formatter.default_time_format 2023-10-31 19:00:46 +03:00
monosans bd5b5db624 Reconfigure ruff 2023-10-28 23:40:44 +03:00
monosans efd842e4cc Reconfigure ruff 2023-10-28 17:41:11 +03:00
monosans 496cef3ae9 Update User-Agent 2023-10-28 14:25:31 +03:00
monosans c849f9a080 Remove dead sources 2023-10-28 01:01:58 +03:00