Enable Custom Robots.txt and Robots Header Tags in Blogger

Hello everyone! Tech Crush Nepal back again with a new article in SEO. Today we'll learn to add custom robots.txt file in blogger.

You may also like-->Meta Tags Generator Tool For Blogger ( SEO )

Robot.txt is a way to tell search engines whether they are allow to index a page in the search result or not. The bots are automatic, and before they could access your site, they check the Robot.txt file to make sure whether they are allowed to crawl this page or not. Sometime, people do not want to stop searching engine from crawling their whole website. On the other hand, they want to specify few pages, which should not be indexed in the search results. Therefore, today in this article, we will show you How to Enable Custom Robot.txt File in Blogger?

First of all, go to Blogger.com >> your site >> Settings >> Search engine Preference >> Crawlers and indexing. Now you will be able to see two options i.e. Custom robots.txt and Custom robots header tags. These two options would provide you the flexibility to customize your robot.txt file.


You may also like-->How to Link 3rd Party Domain(.com.np) in Blogger Blog?

• Custom robots.txt: This option provides you the ability to edit your whole robot.txt file. You can just type your content whether you want the content to be crawled by spiders or not. However, You can always undo your actions and can always revert back to normal. 

• Custom robots header tags: This option is a bit completed. It does not provide you the ability to write your codes instead it provides few options with the check boxes. 

Now you have to enable the custom robot.txt file so press the edit button which is present next to the “Custom robots.txt” option. After selecting the edit button, it would ask you to enable custom robots.txt content so press “Yes” and proceed to the next step.

You may also like-->Overview of Blogger Dashboard

In the text area, type the content which you want to exclude from being crawled.

For Best Result Add This:  Download .txt file

After editing the file according to your needs, press the save button to conclude. However, if you want to revert back to the default robot file then, you can select “No” instead of “Yes. 

Then, to add custom header tags. So go to Settings >> Search preference. There you can see a setting called Custom robots header tags under Crawlers and Indexing. Click the Edit link.

You may also like-->Blogger vs Wordpress (2018) : Blogger Better Than Wordpress?

Tick the boxes accourding to above image for best result. Or, you can use your own if you now well. and save it.

This much for today! I'll be back with a new SEO techniques in upcomming article. Good bye till then. Have a nice time. Peace!