Robots.txt VS. Noindex Robots Meta Tag – When To Use Which?
More info at http://digitalreadymarketing.com/robo... Many people think SEO is all about increasing exposure of websites to search engines by making websites easier to be crawled, increase back links so they will rank higher etc.