What is a Custom Robots.txt File and How to Add it to Blogger

  

What is a Custom Robots.txt File and How to Add it to Blogger?

Following my previous topic, in this article, I will answer questions such as "What is a custom robots.txt file?", "What is it used for?", and "How to add it to Blogger?"

What is a Custom Robots.txt File?

In my previous article titled "What are Custom Robot Header Tags and How to Add Them to Blogger?", I discussed robot header tags and their purposes. Be sure to check that out before reading this. Now, let's move on to the "Custom robots.txt" file. I have received many questions about this, especially from forum sites...

What is a Custom Robots.txt File and How to Add it to Blogger

What is a Custom Robots.txt File and What is it Used For?

As I mentioned in my previous article, a "Custom robots.txt" file is a small code file that contains simple instructions for search engine robots. These codes are stored on our websites or blogs. When our pages are crawled, the robots follow the instructions in this file to crawl and index the pages we want, while ignoring the pages we don't want to be crawled.

By the way, it's important to note that search engine robots scan the Robots.txt file before crawling a website.

How to Add a Robots.txt File to Blogger?

We add the robots.txt file in the Settings section of the Blogger user panel.

Follow the visual guide above. The code we need to add is below: