Robot txt files

Legacy Dynamo
1

can someone explain how to write a simple effective robot txt file

dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

2

Google is your friend

I found this:
http://www.robotstxt.org/orig.html

and

The first one seems fairly clear.

HTH

On Nov 7, 2009, at 7:45 PM, marathon wrote:

can someone explain how to write a simple effective robot txt file

dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

3

Thanks for the info, so I see where to place the name in meta tags–
but what is the best way and how to write the correct format is my
question. Can anyone copy and paste how this should look, thanks!
On Nov 7, 2009, at 2:29 PM, Mike B wrote:

Google is your friend

I found this:
The Web Robots Pages

and

How to Set Up a robots.txt to Control Search Engine Spiders (thesitewizard.com)

The first one seems fairly clear.

HTH

On Nov 7, 2009, at 7:45 PM, marathon wrote:

can someone explain how to write a simple effective robot txt file

dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

4

There are some nice examples at the bottom of this page-
http://www.robotstxt.org/orig.html#examples

If that answers your question
Cheers!

|

On Nov 8, 2009, at 12:51 PM, marathon wrote:

Thanks for the info, so I see where to place the name in meta tags–
but what is the best way and how to write the correct format is my
question. Can anyone copy and paste how this should look, thanks!
On Nov 7, 2009, at 2:29 PM, Mike B wrote:

Google is your friend

I found this:
The Web Robots Pages

and

How to Set Up a robots.txt to Control Search Engine Spiders (thesitewizard.com)

The first one seems fairly clear.

HTH

On Nov 7, 2009, at 7:45 PM, marathon wrote:

can someone explain how to write a simple effective robot txt file

dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

dynamo mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options