Official Google answer: Does Googlebot use inference when spidering - having crawled site.com/article/page1.htm and /page2.htm, can it guess at the existence of a /page3.htm and crawl it?

Question:

Does Googlebot use inference when spidering - having crawled site.com/article/page1.htm and /page2.htm, can it guess at the existence of a /page3.htm and crawl it? Or does it stick entirely to what it finds via the link graph and/or Sitemaps/feeds?

Google's Matt Cutts answers:

Summary of the answer:

The summary of the answer is available to paid members. Become a member now. It's fast, easy and risk-free.

Question	Views
Regarding "nofollow" on internal links: Does it hurt?	598
If Google crawls 1,000 pages/day, Googlebot crawling many dupe content pages may slow down indexing of a large site. In that scenario, do you recommend blocking dupes using robots.txt or is using Meta Robots noindex,nofollow a better alternative?	525
How would Google consider (and rank) a site that uses meta data and URLs in a language (Italian) and has the H1 of the pages in another (English) considered more appealing for users?	443
Tips on requesting reconsideration	651
Links from relevant and important sites have always been a great way to get traffic and acceptance for a website. How do you rate links from new platforms like Twitter, Facebook to a website?	688
Websites lose backlinks due to other websites going out of business or closing (Geocities, AOL member pages). Does Google remove the back link juice that once came from these pages?	470
If I link back to a press article about our company, will Google perceive this as a reciprocal link and therefore give less weight to the inbound link?	392
How does Google calculate site load times in the data it exposes in Google's webmaster statistics?	396
We are a pretty big site. We are changing our hosting company in the next few weeks (same country). Should we be scared from an SEO perspective?	530
How can Googlebot crawl and index pages that don't have any links to them on my website?	511

Question

Views

Regarding "nofollow" on internal links: Does it hurt?

598

If Google crawls 1,000 pages/day, Googlebot crawling many dupe content pages may slow down indexing of a large site. In that scenario, do you recommend blocking dupes using robots.txt or is using Meta Robots noindex,nofollow a better alternative?

525

How would Google consider (and rank) a site that uses meta data and URLs in a language (Italian) and has the H1 of the pages in another (English) considered more appealing for users?

443

Tips on requesting reconsideration

651

Links from relevant and important sites have always been a great way to get traffic and acceptance for a website. How do you rate links from new platforms like Twitter, Facebook to a website?

688