Creativz Lab

Whether you have website or blog or portal, your intention is let the people know about your content.

How to let people know about content on your website or blog or portal. Submitting a sitemap to searchengines helps them to crawl site and present content in their search results. But how better your site is getting indexed or crawled by search engines, depends on how accurate and efficient your sitemap is.

Google provides some tips on how to optimize crawling and indexing.

1) Remove User Specific URLs
– Remove session id’s or user id’s or post id’s from your URLs that don’t change content.
2) Disallow actions that Google bot can’t perform
– Use your robots.txt file to stop crawling of dynamic pages or user entry forms like login page, contact form etc.
3) One to One mapping between URL and Content
–  Make sure URLs point to single piece of content. Never point multiple URLs to same content. This case often
occurs in CMS. Use rel attribute if it is difficult to do so.

Go through following presentation by Google Webmasters to know more.
Optimize Crawling and Indexing
 

Reference: Optimize Crawling and Indexing