haeaeae/proxy-scraper-checker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
315 Commits 1 Branch 0 Tags
df39fd5e00a884d82587c14b659abbeee554b513
Commit Graph
315 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		df39fd5e00 Update .pre-commit-config.yaml (#130) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-30 03:41:14 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		1f998a076f Bump the minor group with 2 updates (#129) 
Bumps the minor group with 2 updates: [aiohttp](https://github.com/aio-libs/aiohttp) and [ruff](https://github.com/astral-sh/ruff).


Updates `aiohttp` from 3.9.2 to 3.9.3
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.9.2...v3.9.3)

Updates `ruff` from 0.1.14 to 0.1.15
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/v0.1.14...v0.1.15)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-30 02:30:27 +00:00
monosans 624ffa2698 Add noqa 2024-01-29 14:04:37 +03:00
monosans 326f143d26 Use Chrome User-Agent 2024-01-29 13:52:46 +03:00
monosans bd1b76a32e Fix aiohttp version 2024-01-29 13:28:55 +03:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		8f4a36c3d8 Bump nuitka from 1.9.7 to 2.0 (#128) 
Bumps [nuitka](https://github.com/Nuitka/Nuitka) from 1.9.7 to 2.0.
- [Changelog](https://github.com/Nuitka/Nuitka/blob/develop/Changelog.rst)
- [Commits](https://github.com/Nuitka/Nuitka/compare/1.9.7...2.0)

---
updated-dependencies:
- dependency-name: nuitka
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-29 04:54:52 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		a1afe51d50 Bump the minor group with 1 update (#127) 
Bumps the minor group with 1 update: [aiohttp](https://github.com/aio-libs/aiohttp).


Updates `aiohttp` from 3.9.1 to 3.9.2
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](https://github.com/aio-libs/aiohttp/compare/v3.9.1...v3.9.2)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-29 02:41:38 +00:00
monosans 2d30886e74 Iterate over ProxyStorage instead of ._proxies 2024-01-29 00:33:55 +03:00
monosans 51c847811c Remove get_sorted method from ProxyStorage 2024-01-29 00:32:31 +03:00
monosans 4aad31cd73 Remove platform from mypy config 2024-01-26 10:43:20 +03:00
monosans e201011ed3 Refactor typing_compat 2024-01-25 18:12:36 +03:00
monosans 330b607c59 Use sys.implementation.name instead of platform.python_implementation() 2024-01-25 15:43:12 +03:00
monosans afb85a6202 Refactor installation & scripts 2024-01-25 14:26:39 +03:00
monosans cabc3a0061 Refactor progress bar 2024-01-25 11:49:22 +03:00
monosans 890c000139 Don't shuffle proxies before checking 2024-01-25 10:30:40 +03:00
monosans 3059940212 Monkeypatch os.link to make aiofiles work on Termux 2024-01-25 09:05:53 +03:00
monosans c481a0e191 Add typing-modules in ruff 2024-01-24 11:42:00 +03:00
monosans 82055d3417 Remove unused variable 2024-01-23 09:49:17 +03:00
monosans dca437feae Add raise_for_status=True when scraping sources 2024-01-23 09:47:43 +03:00
monosans 73303cab1b Improve response decoding performance 2024-01-23 09:37:29 +03:00
monosans 31a50964eb Fix duplicates 2024-01-23 08:09:08 +03:00
monosans 0c0b23a161 Use lto=auto in compilation 2024-01-22 12:13:26 +03:00
monosans 2d6296b577 Enable full extras when compiling 2024-01-22 12:13:18 +03:00
monosans a42448808e Refactor get_grouped 2024-01-21 23:40:05 +03:00
monosans 0984fd1a82 Fix an edge case 2024-01-21 23:37:46 +03:00
monosans d42f09aeba Rename output path 2024-01-21 21:51:31 +03:00
monosans f4af55d37a Fix encoding issues 2024-01-21 21:46:01 +03:00
monosans 24f5eb5f2f Less strict parser 2024-01-21 21:24:09 +03:00
monosans 58935cd328 Remove dead sources 2024-01-21 21:19:33 +03:00
monosans 40b1d1596e Edit README.md 2024-01-21 21:00:43 +03:00
monosansandGitHub e4047e16ae Huge update 
- Drop support for Python 3.7.
- Use offline geolocation database instead of ip-api.com.
- Add support for proxy authentication.
- Add pre-compiled binaries.
- Refactor the code by a lot.
- Simplify config.
- Add automatic text encoding detection.
- Use TOML instead of INI for configuration.
- Add support for using httpbin-compatible services and services which return plain IP address for getting proxy's exit node.
- Add support for reading proxy lists from local files.
- Add support for saving to json.
- Remove support for saving geolocation to txt, use json instead.
- Improve parser.
- Improve config validation.
- Use poetry instead of requirements.txt.
- Use venv in start scripts.
 2024-01-21 17:45:01 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		2415621157 Update .pre-commit-config.yaml (#126) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-20 03:44:32 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		b0e690a466 Update .pre-commit-config.yaml (#124) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-18 03:39:57 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		4f70f33748 Update .pre-commit-config.yaml (#123) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-16 03:41:57 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		64f9a0bc94 Update .pre-commit-config.yaml (#122) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-13 03:40:00 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		3fe4deac44 Update .pre-commit-config.yaml (#121) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-12 03:39:16 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		15311ad964 Update .pre-commit-config.yaml (#120) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-01-03 03:42:00 +00:00
monosans eeca39b8f5 Update User-Agent 2023-12-26 10:09:22 +03:00
monosans 4211933cff Improve traceback 2023-12-25 15:11:23 +03:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		89b1cbe7bc Update .pre-commit-config.yaml (#116) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-23 03:39:42 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		914ae5ee01 Update .pre-commit-config.yaml (#115) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-22 03:41:07 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		640cb5cf6a Update .pre-commit-config.yaml (#114) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-14 03:39:06 +00:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		cea4dffd16 Update .pre-commit-config.yaml (#113) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-11 03:40:25 +00:00
monosans 906bb8387b Reconfigure dependabot 2023-12-10 11:04:27 +03:00
monosans b9a015974e Reconfigure dependency updates 2023-12-10 10:32:29 +03:00
monosans d8179c909d Add instructions for checking local proxy lists 
Closes #22
 2023-12-09 15:36:27 +03:00
monosans 17b680d498 Update sources 2023-12-09 15:27:19 +03:00
monosans cbf10a37d3 Don't set geolocation if geolocation folders are disabled 2023-12-09 15:16:44 +03:00
monosansGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 		d45db44c6b Update .pre-commit-config.yaml (#111) 
* Update .pre-commit-config.yaml

* Ignore PLR0917 in ruff

* Reformat

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-12-07 08:08:12 +00:00
monosans 58292983c8 Change Prettier pre-commit hook 2023-12-02 10:16:20 +03:00