Syncing Data

Overview

There are a few methods involved in syncing data with Merge:

Polling
Webhooks

Merge recommends syncing data in the following way:

Create a set of functions in your backend that are responsible for syncing data
These functions should utilize the modified_after timestamp filter that is available on all Merge list endpoints.
This critical filter enables you to only pull data that has been changed (or created) since your last sync.
For example, you can ask for modified_after=2021-03-30T20:44:18.662942Z, and only pull items that are new / different.
To do this, you need to store the time you last began a fetch to Merge's API for the linked account and endpoint.
Important:
Use the time the fetch began since data can change during a fetch. Pick up those changes during the next sync.
Sync periodically (aka polling)
Configure your backend to call your sync functions periodically, such as every 24 hours.
Sync when Merge emits a Sync Notification Webhook
Merge offers a variety of webhooks for real-time events.
To be alerted to when Merge may have data updates, configure Merge to notify you via webhook when Merge has completed syncing with a third-party platform for a linked account.
Whenever you receive a Sync Notification Webhook, initiate the logic you configured in Step 1.
We recommend using a combination of our "First Sync Notification" and "Any Sync Notification" webhooks to help you manage your sync activities at scale.
Learn how to configure Merge Webhooks in our guide.
I see Merge offers Changed Data Webhooks. Can I use those?
Merge offers webhooks that deliver updated data when individual data instances are updated, but depending on the amount of user data, this will not scale well.
Make sure to implement polling and don't rely entirely on notification webhooks.
They can fail for a variety of reasons (such as downtime on your end or failed processing).
Merge does attempt to redeliver multiple times using exponential backoff, but we still recommend calling your sync functions on a periodic cadence of around once every 24 hours.

Other Important Notes

expandQuery Parameter

Pull multiple models that are related to each other. By default Merge returns just the id. For example, if you are querying for candidates and also want details about associated applications you can expand=applications, and Merge will return the actual application objects instead of just the application_ids. This way, you don't make multiple pulls for what is ultimately related information.

GET /sync-statusEndpoint

When you set up the linking flow, make sure to configure sync-status to set up a trigger so you don't make any API calls before the sync is finished.

Linked Accounts

Next Up

Get Started

API Reference

Syncing Data

Previous

Next Up