Just uploaded my first site and as I finished submitting it to Google Webmaster tools I got an error message asking if robot text file was blocking any pages?? Then, in the crawl errors it said “Google couldn’t crawl your site because we were unable to access robots.txt.”
Any ideas as to what is going on? I havent added a robot file myself.
If you create a blank text file named robots.txt and upload it into the web root on your server with an FTP application, then you will satisfy the 'bot. No idea that they were doing this, an empty file is the same as no file as far as I can tell.
Walter
On May 15, 2012, at 8:23 AM, dunlop1980 wrote:
Hey Guys,
Just uploaded my first site and as I finished submitting it to Google Webmaster tools I got an error message asking if robot text file was blocking any pages?? Then, in the crawl errors it said “Google couldn’t crawl your site because we were unable to access robots.txt.”
Any ideas as to what is going on? I havent added a robot file myself.