Robots.txt Generator

Search Engine Optimization

Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

Robot.txt generator is used to set permission to search engine bots and to tell bots which pages they want to  index and which page it need to leave, otherwise search engines will index all pages of your website including pages you don’t want to be indexed. Search engines will first go through the robot.txt file and then understands which pages are allowed to index and which are excluded from indexing. Free SEO tools provide you the best Robot.txt generator by which you can generate robot.txt file over the web and use it for your website. You just choose the permissions for the search engine bots first then type the folders which you want to resist from bots and finally generate your .txt file and implement it in your website.

When your website doesn’t have a robot.txt file then it shows 404 error when bots come and search for it. Free SEO tool provides the best tool for our beloved users. Robot.txt generator is 100% free, you don’t have to pay, just use it and strengthen your website’s SEO strategies once and for all.