Is there anyway to prevent Google and other search engines from indexing the developmental section of my website? Basically, this is the place where I design and test client’s websites.
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
You can add a robots.txt file to the top level folder below which you don’t want Google to tread. Read more about how to make one and what to put in it here: http://robotstxt.org
Walter
On Mar 8, 2012, at 1:05 PM, RavenManiac wrote:
Is there anyway to prevent Google and other search engines from indexing the developmental section of my website? Basically, this is the place where I design and test client’s websites.
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Thanks Walt. I think the link is actually: http://www.robotstxt.org/
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
So, if I want search engines to index my website, but stay out of a particular directory, like HugeDomains.com, where should I put the robots.txt file?
Also, my clients subdirectory doesn’t contain a homepage or index.html file. Does that matter?
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
On Mar 8, 2012, at 2:17 PM, RavenManiac wrote:
So, if I want search engines to index my website, but stay out of a particular directory, like HugeDomains.com, where should I put the robots.txt file?
It depends on what you put inside that file. If you had it read
User-agent: *
Disallow: /
then you would put that INSIDE the folder with your client’s files.
Also, my clients subdirectory doesn’t contain a homepage or index.html file. Does that matter?
Not really. The kind of Google leakage you are likely to get will come from “REFERER” headers. Let’s say there’s a link to a public side on one of the test pages. When that link is clicked (probably by your client, while reviewing the work) the “REFERER” header is sent to the linked site as part of the request, and if that site included Google Analytics, the existence of your hidden page would become known to the all-seeing eye of Google. So the bot will come a-crawling, and the first thing it will request is the robots.txt file, discover that it isn’t wanted, and move along.
Walter
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Thanks Walter. somebody on the Adobe forums suggested that I password protect my clients directory. What’s the best way to accomplish that?
BTW, I noticed that you use Basecamp to create a client extranet, which looks really cool. Unfortunately, I’m a two person studio so we probably don’t need that level of sophistication quite yet. I do like how professional it is though and one of my biggest problems I have is getting my clients to do something I need (copy, images, etc.) to move their projects forward. I wonder if Basecamp would make that process easier.
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
suggested that I password protect my clients directory. What’s the best way to accomplish that?
Most reasonable hosts offer this facility in their CPanel and it is easily set up there.
Have a look in yours for Password Protect Directories it is usually in the Security Section.
David
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
This is what it looks like in mine http://www.deltadzine.net/cpanel.html
D
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
On Mar 8, 2012, at 2:35 PM, RavenManiac wrote:
Thanks Walter. somebody on the Adobe forums suggested that I password protect my clients directory. What’s the best way to accomplish that?
Your hosting provider’s control panel (cPanel?) will usually have a feature to create “Protected Directories” – I would look there first, as that’s far and away the quickest and easiest way.
BTW, I noticed that you use Basecamp to create a client extranet, which looks really cool. Unfortunately, I’m a two person studio so we probably don’t need that level of sophistication quite yet. I do like how professional it is though and one of my biggest problems I have is getting my clients to do something I need (copy, images, etc.) to move their projects forward. I wonder if Basecamp would make that process easier.
I really like Basecamp, and find that it helps to keep things moving along, particularly on projects where there are multiple stakeholders. It’s a convenient “shared memory” space for everyone to contribute (or just lurk and watch). They have a very inexpensive starting point of $20/month, and it’s pay-as-you-go, so if you decide it’s not adding value, you can quit without penalty.
If you do sign up, please use the recommendation button on my Extranet page, as I get a little kickback for that.
Walter
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
On 8 Mar 2012, 6:44 pm, DeltaDave wrote:
This is what it looks like in mine http://www.deltadzine.net/cpanel.html
D
Thanks Dave. I’ll check it out. Once I’ve password protected my clients directory, how do I give them access to their website? Before I used to simply email them a link, something like www.mycompany.com/clients/client1
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
If you do sign up, please use the recommendation button on my Extranet page, as I get a little kickback for that.
Walter
If I decide to give it a try I’ll absolutely do that. 
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Once I’ve password protected my clients directory, how do I give them access to their website? Before I used to simply email them a link, something like www.mycompany.com/clients/client1
No different but you will need to give them a username and password as well.
You are essentially protecting a folder so while the Clients folder wouldn’t necessarily be protected you could then have an index page within that folder with links to protected sub folders - each with a different password.
ie link to client1 folder (protected), link to client 2 folder (protected) etc.
D
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
Sounds good.
From a workflow prospective, may I ask how you handle that? In other words, do you give each client their own username and password? What does you client access page look like?
Thanks Dave!
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options
do you give each client their own username and password?
Yes - that would be the sensible way to do it. You dont want all clients to have access to all folders - just their own.
D
freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options