Google: disallowed URLs can still appear in search

On Twitter, Google’s John Mueller said that pages that are blocked by the robots.txt file of a website can still appear in Google’s search results if other websites link to these pages. These URLs will be listed in Google’s search results without their contents:

How to find external websites that link to your disallowed pages

You can use the Link Profiler tool in SEOprofiler to find out if external pages link to disallowed pages on your website. Just enter the URL of the web page in the Link Profiler tool and the tool will show you the external pages that link to it:

Analyze external links

Tom Cassy

Tom Cassy is the CEO of SEOprofiler. He blogs about search engine optimization and website marketing topics at “http://blog.seoprofiler.com”.