1. How often will Google crawl my site?

Google's spiders regularly crawl the web to rebuild our index. Crawls are based on many factors such as PageRank, links to a page, and crawling constraints such as the number of parameters in a URL. Any number of factors can affect the crawl frequency of individual sites.

Our crawl process is algorithmic; computer programs determine which sites to crawl, how often, and how many pages to fetch from each site. For tips on maintaining a crawler-friendly website, please visit our Webmaster Guidelines.

2. How can I migrate my site to a new IP address?

We recommend migrating a site to a new IP address with the following steps:

1. Bring a copy of your site up at the new IP address.
2. Update your nameserver to point to the new IP address.
3. Once you see search engine spiders fetch pages from the new IP address (typically within 24-48 hours), it's safe to take down the copy of your site at the old IP address.

3. Why is my site labeled "Supplemental"?

Supplemental sites are part of Google's auxiliary index. We're able to place fewer restraints on sites that we crawl for this supplemental index than we do on sites that are crawled for our main index. For example, the number of parameters in a URL might exclude a site from being crawled for inclusion in our main index; however, it could still be crawled and added to our supplemental index.

The index in which a site is included is completely automated; there's no way for you to select or change the index in which your site appears. Please be assured that the index in which a site is included does not affect its PageRank.

4. I'd like my site to return for pages from a specific country.

While all sites in our index return for searches restricted to "the web," we draw on a relevant subset of sites for each country restrict. Our crawlers may identify the country for a site by factors such as the physical location at which the site is hosted, the site's IP address, the WHOIS information for a domain, and its top-level domain.

That said, your site's top-level domain doesn't need to match the country domain for which you'd like it to return. It's also important to keep in mind that our crawlers don't index duplicate content, so creating identical sites at several domains will likely not result in their returning for many country restricts. If you do create duplicate domains, we suggest using a robots.txt file to block our crawler from accessing all but your preferred one.

1. What is "results prefetching," and how does it impact my site?

On some searches, Google uses a special tag supported by Firefox and Mozilla to instruct the browser to download the top search result before the user clicks on the result. When the user clicks on the top result, the destination page will load faster than before. This tag is only inserted when it is likely that the user will click on the first link.

For example, when a Firefox user searches for [stanford], Google includes the following tag in the results HTML:

<link rel="prefetch" href="http://www.stanford.edu/">

The official Mozilla Link Prefetching FAQ describes the behavior of this tag in detail.

Prefetching may impact your site because the prefetch request will happen whether or not the user clicks on the result, so it may result in additional traffic to your web server. Google only inserts this tag when there is a high likelihood that the user will click on the top result, but clearly this heuristic is not right 100% of the time.

2. Can I distinguish prefetch requests from normal requests?

Yes, as described in the Mozilla Link Prefetching FAQ, prefetch requests include the additional HTTP header

X-moz: prefetch

3. I want to block/ignore prefetch requests. What should I do?

To block or ignore prefetch requests (from Google and other web sites), you should configure your web server to return a 404 HTTP response code for requests that contain the "X-moz: prefetch" header.