What is a Robots.txt File?

It’s a small text file that tells Google what it should or should not put in it’s search results.

Why You Need a Robots.txt File

The search engines like it when you have a robots.txt file. It tells the search engines what you want being indexed (listed) in the search results.

This is How You Make a Robots.txt File

1. Open notepad
2. Copy and paste the below bolded text into the empty notepad file

User-agent: *Allow: /
Disallow: thankyou.html

3. Save it as robots.txt
4. Upload it to the root directory of your hosting account
5. You are done!

For everything you don’t want someone to or the search engines looking up (i.e. = not relevant to your website), type new Disallow line for each thing, like this…

User-agent: *Allow: /
Disallow: thankyou.html
Disallow: /myporncollection
Disallow: /this-old-web-page-i-dont-want-anyone-seeing.html
Disallow: /thispornvideo.mpeg

That is how you make a robots.txt file

If you are using wordpress, I would suggest downloading this XML Generator WordPress Plugin.

On 9th March 2012  /   Tips From Our Team  /   Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *

one + 4 =