Getting indexed by Google fast is an opportunity not many websites are exposed to since the majority of them rely on a sitemap. The XML sitemap does help Google discover your pages, but it is not fast enough to display relevant changes in search.
Thus, if you want to see your changes reflected in Google search results almost instantly, you need Google Indexing API. It is a speedier and more direct approach to indexing that outperforms standard crawling.
While it is relatively hard to set up, you can still do it by using the Magento 2 Google Indexing API Extension. In this guide, though, you'll learn all about the prerequisites of the Indexing API and how to set it up for Magento.
Post Contents [hide]
Before we dive deeper into the topic, let's define what is Google Indexing API.
What is Google Indexing API?
Google Indexing API is a powerful tool that allows you to notify Google when pages on your website are added, updated or removed. Correspondingly, Google schedules a fresh crawl each time you make changes to your website pages. As a result, those changes are reflected in the search results faster.
The Indexing API gives you more control over what gets indexed and when. Nonetheless, Google still recommends you submit the XML sitemap as a "backup".
Note: Google Indexing API has a limit of 200 requests per day. If need to send more, increase the Google Indexing API quota.
Why Do You Need Google Indexing API?
Although we have partially answered this question, let's summarise the things you can do with the indexing API.
- Update URL — notify Google when you create a new page or when you update a page you've submitted before.
- Remove URL — notify Google to remove a page from search results, so Google doesn't attempt calling it if the page is deleted.
- Update/remove URLs in bulk — send indexing requests to Google in batches.
- Get URL index status — check when Google received any indexing notifications for a certain page.
Now, it's obvious that using Google indexing API gives you leverage. You don't have to wait for Google to discover your pages through the sitemap or submit each page for indexing in the Google Search Console (GSC).
How to Set Up Google Indexing API in Magento?
Now it might seem that the Google Indexing API is too complex to set up. Well, it could be. However, if you follow the detailed steps we describe below, you can still do it without much effort.
Magento 2 Google Indexing API extension does all the heavy weightlifting — sending the requests via API. All you need to do is set up a project in Google Cloud and create a service account, if you haven't done it already.
Let's take one step at a time.
Step 1: Create a project
You need to register your application and activate access to the API before you can send requests to the Indexing API. You do that in the Google API Console, which guides you through the process.
Note: if you haven't worked with Google Cloud before, you'll be asked to confirm the Terms of Service.
First, navigate to the Google Cloud Console and click the Create Project button.
You need to fill out the Project name, choose Organization and set a Location. Click Create once you fill out the required data.
Note: an organization is a resource representing a company or organization and is a hierarchical super node of Google Cloud Projects. It's fine if you don't choose any organization when creating a project. However, you can read more about the organization in Google documentation.
Then, you are redirected to the dashboard where you need to Confirm the project. Simply click on the Next button.
And finally, Enable the API by clicking on the Enable button.
That's it, you've created a project.
Step 2: Create a service account
Go to the Service account page in the Google Cloud and choose your project if prompted.
Then click on the +Create Service Account button.
You need to fill out the following details and click Done:
- Service account name
- Service account ID
- Service account description
Note: additionally, you can grant this service account access to the project and allow certain users access to this service account. Both are optional, so you can just skip them.
Once you create a service account, you need to create keys. For that, click on the three dots under Actions and choose the Manage keys option.
In Keys, click on the Add key option to Create a new key.
Choose JSON as a private key format and click Create.
This will immediately download the JSON file with private keys to your device. We'll need them later.
Step 3: Add a service account as a site owner
As you might know, indexing is done through Google Search Console. So, to continue working with the Indexing API, you need to verify you're the website owner and add a service account as the owner.
You have probably completed the verification step already if you use the XML sitemap for your Magento website. Google offers multiple verification methods you choose when setting up the GSC for your store.
The only thing left is to grant owner status to the service account we've just created. For that, go to the Google Search Console settings > Users and permissions and click on the Add User button.
Then specify the Email Address of the user and give them the Owner Permission.
Note: open the JSON file you've retrieved before and find the email in the client_email
field.
Once you add a new user, you'll see it in the users list.
Important: if you have a multi-website Magento setup, you need to add the service account as the owner for each Google Search Console property. This will then trigger indexing requests for the page URL on each website.
Step 4: Upload the private key to Magento
The last step you need to take to set up Google Indexing API for Magento is to add your JSON file to Magento.
Navigate to Stores > Configuration > Magefan Extensions > Google Indexing and make sure the extension is enabled.
Then, move to the Credentials section and upload the Private Key JSON File.
Click Save once you upload the file, and that's it. You can now submit Magento URLs for indexing individually or in batches. However, before you send the indexing requests to Google, configure the Google Indexing API Extension for the optimal experience.