[Pro] Disallow development server?

How do I disallow all of the websites on my development server if the only way you access said sites is with a subdomain redirect that bypasses the root?

At present I have a robots.txt file in the root folder with the following code:

User-agent: *

Disallow: /


offtopic mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Are you looking to avoid people being able to browse to www.example.com/subdomain and force them to go to subdomain.example.com? If so, you want an htaccess redirect, not a robots.txt. The robots file is purely for well-behaved spiders, and does nothing for normal browsers or spiders that are not well-behaved.

Google: mod_rewrite redirect to subdomain

Walter

On Feb 12, 2013, at 10:59 AM, RavenManiac wrote:

How do I disallow all of the websites on my development server if the only way you access said sites is with a subdomain redirect that bypasses the root?

At present I have a robots.txt file in the root folder with the following code:

User-agent: *

Disallow: /


offtopic mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


offtopic mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

No, I just don’t want search engines to index my internal server.


offtopic mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Then what I would recommend is that you use your cPanel (or local equivalent) to set a simple password on that folder. Tell all your clients what it is. That will stop all crawlers cold, good and evil. Even better is to have a separate password for each client’s folder. It doesn’t have to be a serious password (it will be sent in plain text anyway) but just the fact that it is there will keep the honest folk honest, and will keep the bots out entirely.

Walter

On Feb 12, 2013, at 11:50 AM, RavenManiac wrote:

No, I just don’t want search engines to index my internal server.


offtopic mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


offtopic mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options