A test on slowing the Google Crawl Rate

Last week I decided to change the crawl rate of one of the sites I control within the Google Webmaster Tools. I am left with mixed results, so far after one week of changing this switch there has not been a significant decrease in crawl traffic from what is normally seen. Admittedly the day I turned it down Google was crawling twice as much as any normal day, it is still crawling at a higher than average level.

Just after this feature was launched, I tried the opposite on a new site that was launched around the same time. I set the crawl rate to faster which has had limited to no effect on the crawl rate.

With this data I believe that this feature may be live on the Webmaster Tools, however it does not have enough of an overriding strength to determine Google’s crawl rate at this time. I would bet that they are still experimenting with what this tool is capable of doing. I will update you as soon as I see a change.

Crawl Rate just appeared in my Google Webmaster Tools

I have been trapped in meetings all week so I have not posted, hope to post more next week.

I am so excited I finally have the crawl rate control in my Google Webmaster Tools, what a great feature! My sites get millions of crawls a data from Google’s data centers and it’s amazing to see how much capacity it sucks up. But now when user traffic spike I can ask Google to slow down for a while to keep the load off of my sites. What a great tool