Google provides each hostname its personal allotted crawl price range. So which means every area, subdomain, and so forth has its personal distinctive crawl price range.
This was introduced up within the feedback are in a LinkedIn put up the place Tony McCreath requested a Google crawl price range questions associated to crawling a web page on area A that masses a useful resource from area B (totally different server).
John Mueller from Google responded, saying, “They’re unbiased.”
Then former Googler, Pedro Dias wrote, “Afaik, it is allotted set by hostname, proper? So, even inside the similar area, a number of subdomains might need their very own crawl-rate limits.”
Gary Illyes from Google confirmed this assertion to be true, replying, “appropriate.”
Right here is the put up:
Need to be taught extra about crawl price range, see this Google help document.
Discussion board dialogue at LinkedIn.
