How To Make Google Instantly Crawl Your Site and Increase its Visibility

api webdev php googlecloud

In this article, we'll look at how to directly notify Google of changes to the site and share our experience using the Google Indexing API with examples, to solve page indexing problems that prevent robots from getting up-to-date site information.

Overview

To address this issue, Google developers have created the Indexing API, which allows the Googlebot to check your pages before the sitemap is updated.

The Indexing API allows you to control actions such as:

  • Update URL - when the page content has been updated.
  • Remove URL - when the page no longer exists or is not available.
  • Getting the status of the last indexation

The quota for sending data to Google is 200 links/day, but you can also increase the quota.

Benefits of using Indexing API

  • Content actualization. This is very important when the information on the page changes frequently.
  • Fast ranking of the site directly affects the search results and what position your site will take.

Step-by-step instructions for setting up the Google Indexing API

On the example of one of our sites we'll show the algorithm of all actions to activate quick indexing of your site. Let's take dogsrank.com as an example.

Project creation and API activation

1.Let's go to Google Cloud Platform and create a new project for our site.

Creating a new Google Cloud Platform project

2.Select the previously created project and activate the Indexing API for it.

Enabling the Indexing API for the dogsrank project. Step 1

Enabling the Indexing API for the dogsrank project. Step 2

Google does not charge a fee for using this API.

Creating a service account

1.Go to the page with service accounts and choose our project.

Creating a service account

2.Click the Create Service Account button. The name of the service account can be arbitrary. Then we will need it for communication with the domain and access rights to it.

Service Account Details

3.It is not necessary to set the rights settings. Just click Done. 4.Copy the Email of the service account to add it to Search Console.

List of created service accounts for the selected project

5.Go to the KEYS tab of the previously created service account and create a JSON key to access the API. A key pair (public and private) will be created, which will then be downloaded to your computer.

Creating a private key for API access

6.Save the downloaded .json for later use.

List of created keys for the selected project

Adding a service account to the owners of the site

  1. Go to the Search Console settings page and add the previously created service account to the site owners. Click Add User.

Service account email

This article assumes that the domain has already been validated through Search Console.

Seach Console adding a user

Getting an access token and sending links to update indexing

We have successfully created a project, a service account and generated access tokens. Now it's time to try everything in action.

For example, we will use the PHP language and API library.

GitHub package for working with Google Indexing API

  1. Load to the configs folder the config that we created at the stage of creating a key for the service account.
  2. Create a file .env and write the name of the variable GOOGLE_AUTH_CONFIG with the same name in the configs folder.
  3. Insert all the links you want to update into /src/urls.txtfile. Add https://dogsrank.com/ to the beginning of file.
  4. Open /src/index.php. On line 18 we see that we call the updateUrl method, which will tell Google to index the link.
  5. Run the script php index.php

Example API response

 % php index.php 
Indexing URL: https://dogsrank.com/
200
{
  "urlNotificationMetadata": {
    "url": "https://dogsrank.com/",
    "latestUpdate": {
      "url": "https://dogsrank.com/",
      "type": "URL_UPDATED",
      "notifyTime": "2022-12-04T21:55:33.449222083Z"
    }
  }
}

Related posts

Similar Articles