Stand with 
Many store owners falsely believe that the more pages they have on their website and Google indexing, the more traffic they receive. It is a common misconception that can even hurt your Magento SEO. Especially if not all the pages contain high-quality content.
Thus, sometimes you might want to remove pages from Google indexing. Whether it's existing pages or future URLs, there is a certain process you should follow.
We've got you covered. You'll learn everything you need to remove Magento pages from the Google index here.
Post Contents [hide]
Why Do You Need To Remove Pages from Google Indexing?
There are multiple reasons why you might not want a page to appear in search results. Nonetheless, you should know about the most common ones.
Duplicate Content
One of the most common cases why you need to remove a page from the Google index is when you have several of them. It often happens if you have two similar pages, but one of them is used for a Google Ads campaign.
Although the Google Ads page's focus is on generating conversion, it still reuses existing content. This correspondingly could cause duplicate content issues. Even though it's an unlikely scenario, you still wouldn't want the Google Ads page to appear in organic search results.
Private Web Pages
Depending on the customer groups and CMS page rule you have, there might be content you only display for logged-in users or premium partners. In this case, it's also best to remove such URLs from the Google index.
Development/Testing
Finally, if you have some preview or development version of the page on your store, you wouldn't want it to appear along the live URLs, would you?
Remove URLs from Google Index Using GSC
If you want to remove some pages from search results, Google provides native tools for that — Google Search Console (GSC). The only downside is, it works only for 6 months.
Navigate to your Google Search Console panel and choose a property you'd like to stop Google from indexing.
Move to Removals from the left-hand menu and click on the New Request button.
Specify the URL you'd like to remove from Google search and choose the Remove this URL only option.
Note: if you want to remove every URL within the website (domain) select the Remove all URLs with this prefix option.
Submit the request and check your URL in the temporary removals.
Again, GSC will only remove the page from the Google index for 6 months. So you might still need to resort to other solutions we provide here.
Remove Pages from Google Index Using Robots.txt
As you might know, Magento robots.txt is a file you use to provide instructions for search robots. It is a file where you disallow search crawlers from indexing certain files, folders and pages. So if you want to remove some pages from indexing, add the page path to the robots.txt file, as in the following example:
Disallow: /customer/
Note: the page you disallow indexing in robots.txt is musn't be included in the website sitemap.
Unfortunately, this method doesn't always work.
Google can discover your website URL through the sitemap you submit to GSC or through the index you require in the GSC.
The other way is when Google follows a link from another website to yours. That's the case when Google may simply not see your robots.txt file to learn what pages not to index. As a result, some pages will still appear in search.
Remove Pages from Google Index Using Meta Robots
This is probably the easiest and the most beginner-friendly way to remove pages from Google Index. You can simply add NOINDEX NOFOLLOW tags in the <head> tag of your page through the admin panel.
Start with going to the page you'd like to remove from the search in the backend (Content > Pages). Then find the Design section and edit the Layout Update XML field.
Add the following code there:
<head>
<meta name="robots" content="NOINDEX,NOFOLLOW"/>
</head>
Hit Save and check the robots on the frontend:
The page you've added these meta robots to will be removed from Google indexing. However, you will have to do it for every single page, which is kind of daunting.
So, if there is a set of pages that share similar features, there is a simpler way — Magento SEO Rules. They allow you to add NOINDEX NOFOLLOW meta robots for multiple pages at once.
For that go to Marketing > SEO by Magefan > SEO Rules and start creating a new rule. Fill out general rule information, and choose the type of page you want to create a rule for.
Then move to the Meta Robots opinion and choose a corresponding option from the dropdown.
Additionally, you can set the product attribute. The rule will use it to identify what pages to add these meta robots to.
Don't forget to Save the rule to apply it. And that's how you add meta robots for multiple pages in bulk.
Remove Pages from Google Index immediately
Even if you set the custom robots and block a page in robots.txt, it might still be displayed in search results. So, to remove pages from index immediately, simply configure Google Indexing API. It allows you to remove individual or multiple pages from the Google index at once.
Nonetheless, there is no one-size-fits-all answer when it comes to removing pages from Google indexing. You should be careful with whatever method you choose since indexing is a crucial step in your SEO strategy.
