-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Not working when deployed (Google Cloud). TimeoutError: waiting for selector .cf-browser-verification
to be hidden failed: timeout 30000ms exceeded
#40
Comments
Waiting for I have no way around that currently, deploying scrapers on cloud infrastructure is difficult. If you (or someone else) has ideas please share, its a very common problem. One solution that might work but is costly is using a service like bright data (proxy with unblocker API). |
.cf-browser-verification
to be hidden failed: timeout 30000ms exceeded.cf-browser-verification
to be hidden failed: timeout 30000ms exceeded
UPDATE: When running on GCP we now have a less frequent
I hope this will be fixed by v7's new approach 🤞 |
REPORT FROM @mlarcher : I dug a bit into the code and setup a test case...
:( From what I gathered :
All in all this doesn't seem too good, but not directly related to the current library. Let me know if you have expertise on the matter and know some other way to tackle the problem though :) |
Bypassing cloudflare is definately not my expertise. I have tried to solve this problem for some time now, and it is definately possible but as you mentioned its an arms race. I tried these packages:
|
This is what I get running on GCP using offersByScrolling:
TimeoutError: waiting for selector ``.cf-browser-verification`` to be hidden failed: timeout 30000ms exceeded
It seems it is sometimes working and sometimes failing on this error.
Any idea what's happening there ?
The text was updated successfully, but these errors were encountered: