haeaeae/proxy-scraper-checker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
461 Commits 1 Branch 0 Tags
33daaec628fb198d88d56592923ea60cd3b85bd2
Commit Graph
113 Commits
Author SHA1 Message Date
monosans ef3ca6f304 Refactor run function 2024-05-31 21:12:18 +05:00
monosans 1372ab308b Fix mypy 2024-05-31 13:48:09 +05:00
monosans 82b097c6a6 Add async_run function instead of set_event_loop_policy 2024-05-31 12:30:17 +05:00
monosans 6de2dc8a2c Add type: ignore 2024-05-06 23:03:54 +05:00
monosans c487ae8c23 Add Accept header for httpbin 2024-05-06 22:55:17 +05:00
monosans 59a98e1236 Add type: ignore 2024-05-06 22:53:44 +05:00
monosans fdae372a08 Update User-Agent 2024-05-06 22:53:34 +05:00
monosans 00cffb5605 Support IPv6 on httpbin.org 2024-04-20 13:00:44 +05:00
monosans dbdcea6d86 Improve check_website validation error message 2024-04-11 23:13:31 +03:00
monosans 0050b08d87 Make sure check_website has an http or https scheme 2024-04-11 23:12:00 +03:00
monosans 186cadc764 Check that the source scheme is http or https when deciding whether it is a URL or a local file 2024-04-11 22:57:18 +03:00
monosansandGitHub 74edec2330 Support older winloop 2024-04-03 20:07:11 +00:00
monosansandGitHub 98d023c289 Add winloop support, remove max_connections limit on Windows 2024-04-03 19:58:58 +00:00
monosans f825a801ac Fix httpbin.io 2024-03-26 16:45:19 +03:00
monosans df408d62b7 Update User-Agent 2024-03-23 09:53:34 +03:00
monosans 38f910d1a6 rstrip instead of strip 2024-03-23 09:47:55 +03:00
monosans 5ded527f73 Fix possible KeyError 2024-03-23 09:47:19 +03:00
monosans dbdf5b84be Fix 2024-03-01 07:28:43 +03:00
monosans 2de104df7d Create cache dir only if geolocation is enabled 2024-02-29 22:29:12 +03:00
monosans 32fd6158ec Remove ETag from cache if response didn't provide it 2024-02-18 12:11:04 +03:00
monosans 8ed0cf378d force=True in logging.basicConfig 2024-02-13 21:45:41 +03:00
monosans 75fca04801 Rename argument 2024-02-08 12:01:37 +03:00
monosans dbeeec5641 Refactor asyncify 2024-02-08 11:22:51 +03:00
monosans 91d2624224 Remove maybe_add_permission 2024-02-08 09:42:30 +03:00
monosans 2cca716167 Remove monkeypatch 2024-02-08 09:27:42 +03:00
monosans a42f44e7e8 Automatically fix permissions issues 2024-02-08 09:12:13 +03:00
monosans 95c6e7f6eb Reformat utils.py 2024-02-07 13:03:31 +03:00
monosans 04bf2f6850 Fix directories access check 2024-02-07 12:37:46 +03:00
monosans 35518b67e9 Reformat 2024-02-07 10:25:01 +03:00
monosans 9026cf9fb4 Fix ruff 2024-02-07 10:21:22 +03:00
monosans d57620cae4 Minor improvements 2024-02-04 16:58:45 +03:00
monosans 419d9fc036 Fix AttributeError 2024-02-04 12:24:12 +03:00
monosans 328834eb03 Improve permissions check 2024-02-01 14:52:29 +03:00
monosansandGitHub 5005e0dc6b Add Docker support 2024-02-01 10:24:26 +03:00
monosans 624ffa2698 Add noqa 2024-01-29 14:04:37 +03:00
monosans 326f143d26 Use Chrome User-Agent 2024-01-29 13:52:46 +03:00
monosans 2d30886e74 Iterate over ProxyStorage instead of ._proxies 2024-01-29 00:33:55 +03:00
monosans 51c847811c Remove get_sorted method from ProxyStorage 2024-01-29 00:32:31 +03:00
monosans e201011ed3 Refactor typing_compat 2024-01-25 18:12:36 +03:00
monosans 330b607c59 Use sys.implementation.name instead of platform.python_implementation() 2024-01-25 15:43:12 +03:00
monosans cabc3a0061 Refactor progress bar 2024-01-25 11:49:22 +03:00
monosans 890c000139 Don't shuffle proxies before checking 2024-01-25 10:30:40 +03:00
monosans 3059940212 Monkeypatch os.link to make aiofiles work on Termux 2024-01-25 09:05:53 +03:00
monosans 82055d3417 Remove unused variable 2024-01-23 09:49:17 +03:00
monosans dca437feae Add raise_for_status=True when scraping sources 2024-01-23 09:47:43 +03:00
monosans 73303cab1b Improve response decoding performance 2024-01-23 09:37:29 +03:00
monosans 31a50964eb Fix duplicates 2024-01-23 08:09:08 +03:00
monosans a42448808e Refactor get_grouped 2024-01-21 23:40:05 +03:00
monosans 0984fd1a82 Fix an edge case 2024-01-21 23:37:46 +03:00
monosans f4af55d37a Fix encoding issues 2024-01-21 21:46:01 +03:00