I talked to my hosting today and they say that the Google bot is not a problem
but other search engines can block a site totally and can be compared to DDoS attacks.
Suggestion was to test crawl delays up to 30 seconds as these bad bots harvest a site...
but other search engines can block a site totally and can be compared to DDoS attacks.
Suggestion was to test crawl delays up to 30 seconds as these bad bots harvest a site...