How to test a robots.txt file using Google Webmaster tools
Google have introduced a wonderful tool in webmaster tools on Wednesday called as robots.txt file checker that checks out for errors in your robots.txt file and suggests you modifications that are needed.
Read full article: http://goo.gl/aSUYkV