Concept of Synchronization
For proper synchronization of your resources is important to understand the principles of Google API. These principles work the same for all Google resources, but will be different for Outlook. We will figure out how and why to use query parameters and study the best practices.
For optimized performance the API uses for important parameters as syncToken and pageToken.
The API data in most cases is returned with pagination, so as not to burden the network and allocate resources on the network and their caching.
Resource pagination - pageToken
When there is more than one page in the response, you can see the nextPageToken field, which stores the data received about the next page.
Do not forget to save the
nextPageTokenfield in case you get an error when synchronizing one of the pages and do not want to retrieve successfully saved resources, but only starting from a certain page.
When you want to get the next page of data, you must specify the pageToken with the value of nextPageToken. You don't need to send additional parameters, because the token already has everything.
An example looks like this:
1. GET /calendars/primary/events
// Response
"items": [...]
"nextPageToken":"CiAKGjBpNDd2Nmp2Zml2cXRwYjBpOXA"The following query takes the value from nextPageToken and sends it as a value for pageToken.
2. GET /calendars/primary/events?pageToken=CiAKGjBpNDd2Nmp2Zml2cXRwYjBpOXA
You can control the number of displayed resources in the response through the maxResults parameter.
Synchronization Marker - nextSyncToken
During the first synchronization, an initial query is performed for each resource in the collection that you want to synchronize.
The synchronization token is represented as a field named nextSyncToken in the list operation response.
The nextSyncToken is an important field for optimizing the synchronization of your resources, saving bandwidth. It allows you to retrieve only new data from when the token was first issued.
Don't forget to save this
nextPageTokento retrieve resource items from the last received page.
For example, if you create a new event in your calendar, you don't need to retrieve the whole list of events and check and process each one, instead you get only the updated data.
An example looks like this:
1. GET https://www.googleapis.com/calendar/v3/users/me/calendarList
// Response
...
"items": [...]
"nextSyncToken": "CPDAlvWDx70CEPDAlvWDx70CGAU=",The nextSyncToken field will be only in the response on the last page, because all requests are given on a page-by-page basis and will contain the nextPageToken parameter.
The following query takes the value from nextSyncToken and sends it as a value for syncToken.
2. GET https://www.googleapis.com/calendar/v3/users/me/calendarList?syncToken=CPDAlvWDx70CEPDAlvWDx70CGAU=
// Response
...
"items": [...]
"nextSyncToken": "v7GC9pHgvO6kpTHAxRx71KebukwS=",In cases where your
syncTokenis no longer valid, you should remove it from the database and re-request the entire resource collection.
Synchronizing Google calendars
In the previous post, we set up authorization via oauth2, after which the Google Account data is written to the database.
After successful authorization, you should get the list of available calendars of the user from the account.
public function callback(string $driver): RedirectResponse
{
/** @var ProviderInterface $provider */
$provider = $this->manager->driver($driver);
/** @var Account $account */
$account = $provider->callback();
$accountId = app(AccountService::class)->createFrom($account, $driver);
$account->setId($accountId);
// Sync calendars of user account
$provider->synchronize('Calendar', $account);
return redirect()->to(
config('services.' . $driver . '.redirect_callback', '/')
);
}For calendars records and their basic information let's design a table in the database, which looks like:
Schema::create('calendars', function (Blueprint $table) {
$table->id();
$table->string('summary')->nullable();
$table->string('timezone')->nullable();
$table->string('provider_id');
$table->string('provider_type');
$table->text('description')->nullable();
$table->text('page_token')->nullable();
$table->text('sync_token')->nullable();
$table->timestamp('last_sync_at')->nullable();
$table->boolean('selected')->default(false);
$table->unsignedBigInteger('account_id');
$table->foreign('account_id')->references('id')->on('calendar_accounts')->onDelete('CASCADE');
$table->index(['provider_id', 'provider_type']);
$table->timestamps();
});The table will store information about the calendar, tokens for synchronization and pagination for events, as well as links to his account.
To perform synchronization, add some logic to our calendar driver - GoogleProvider.php.
public function synchronize(string $resource, Account $account, array $options = [])
{
$resource = Str::ucfirst($resource);
$method = 'synchronize' . Str::plural($resource);
$synchronizer = $this->getSynchronizer();
if (method_exists($synchronizer, $method) === false) {
throw new \InvalidArgumentException('Method is not allowed.', 400);
}
return call_user_func([$synchronizer, $method], $account, $options);
}The getSynchronizer() function will return us the synchronizer class, which will mediate the resources. Which has method: synchronizeCalendars().
public function synchronizeCalendars(Account $account, array $options = [])
{
$token = $account->getToken();
$accountId = $account->getId();
$syncToken = $account->getSyncToken();
if ($token->isExpired()) {
return false;
}
$query = array_merge([
'maxResults' => 100,
'minAccessRole' => 'owner',
], $options['query'] ?? []);
if (isset($syncToken)) {
$query = [
'syncToken' => $syncToken,
];
}
$body = $this->call('GET', "/calendar/{$this->provider->getVersion()}/users/me/calendarList", [
'headers' => ['Authorization' => 'Bearer ' . $token->getAccessToken()],
'query' => $query
]);
$nextSyncToken = $body['nextSyncToken'];
$calendarIterator = new \ArrayIterator($body['items']);
/** @var CalendarRepository $calendarRepository */
$calendarRepository = app(CalendarRepository::class);
// Check user calendars
$providersIds = $calendarRepository
->setColumns(['provider_id'])
->getByAttributes(['account_id' => $accountId, 'provider_type' => $this->provider->getProviderName()])
->pluck('provider_id');
$now = now();
while ($calendarIterator->valid()) {
$calendar = $calendarIterator->current();
$calendarId = $calendar['id'];
// Delete account calendar by ID
if (key_exists('deleted', $calendar) && $calendar['deleted'] === true && $providersIds->contains($calendarId)) {
$calendarRepository->deleteWhere([
'provider_id' => $calendarId,
'provider_type' => $this->provider->getProviderName(),
'account_id' => $accountId,
]);
// Update account calendar by ID
} else if ($providersIds->contains($calendarId)) {
$calendarRepository->updateByAttributes(
[
'provider_id' => $calendarId,
'provider_type' => $this->provider->getProviderName(),
'account_id' => $accountId,
],
[
'summary' => $calendar['summary'],
'timezone' => $calendar['timeZone'],
'description' => $calendar['description'] ?? null,
'updated_at' => $now,
]
);
// Create account calendar
} else {
$calendarRepository->insert([
'provider_id' => $calendarId,
'provider_type' => $this->provider->getProviderName(),
'account_id' => $accountId,
'summary' => $calendar['summary'],
'timezone' => $calendar['timeZone'],
'description' => $calendar['description'] ?? null,
'selected' => $calendar['selected'] ?? false,
'created_at' => $now,
'updated_at' => $now,
]);
}
$calendarIterator->next();
}
$this->getAccountRepository()->updateByAttributes(
['id' => $accountId],
['sync_token' => Crypt::encryptString($nextSyncToken), 'updated_at' => $now]
);
}The code above fetch list of user calendars where it has owner access. After each calendar is checked for consistency in the database and actions are taken to delete, update or create.
As a result, we remember the synchronization token for the current account in the database.
Code Notes
- Refresh token will be implemented in the following articles.
- Google API return 100 calendars by default. Let's omit the point that we include more.
- We get calendars where the user can read and modify events and access control lists.
- The presence of
syncTokenin the request does not allow other parameters. Throws exception. nextSyncTokenis encoded before writing to the database.
Performance tips
To get a gzip-encoded response, you need to do:
- Set the Accept-Encoding header
- Change your user agent so that it contains the gzip string.
Each request to api google will contain these headers.
protected function headers(array $headers = []): array
{
return array_merge([
'Content-Type' => 'application/json',
'Accept-Encoding' => 'gzip',
'User-Agent' => config('app.name') . ' (gzip)',
], $headers);
}Sync calendar command
To test the synchronization result, we will use the commands in Laravel. let's call it synchronize:calendars.
Commands will retrieve all accounts from the database and still synchronize the list of calendars.
public function handle()
{
/** @var GoogleProvider $provider */
$provider = app(CalendarManager::class)->driver('google');
$accounts = app(AccountRepository::class)->get();
foreach ($accounts as $accountModel) {
$provider->synchronize('Calendar', tap(new Account(), function ($account) use ($accountModel) {
$token = Crypt::decrypt($accountModel->token);
$syncToken = '';
if (isset($accountModel->sync_token)) {
$syncToken = Crypt::decryptString($accountModel->sync_token);
}
$account
->setId($accountModel->id)
->setProviderId($accountModel->provider_id)
->setUserId($accountModel->user_id)
->setName($accountModel->name)
->setEmail($accountModel->email)
->setPicture($accountModel->picture)
->setSyncToken($syncToken)
->setToken(TokenFactory::create($token));
}));
}
}Bottom line
In this article, we looked at how to sync google calendars to your Laravel app. We looked at the basic parameters for getting api resources. Created code and optimized it.
Some aspects of the implementation were omitted, but the entire list of changes can be seen at the link.
In the next article we will see how to synchronize google events.
