Real-time Sync of Salesforce Data using Change Data Capture (Pilot)

New feature of Winter'18 release -

The purpose of this article is to share how can we receive near-real-time changes of Salesforce records, and synchronize corresponding records in an external data store.

What is CDC (Change Data Capture Pattern)?

Change Data Capture as the name suggests is a design pattern to track changes in data so that an action can be taken based on the change. Changes include creation of a new record, updates to an existing record, deletion of a record, and undeletion of a record. CDC is common in most of the ETL tools however it is design pattern to track any data changes in any data repository that could be data warehouse or data bases. Data change events are published on a channel (the event bus) when changes occur in Salesforce.

For example, you have a human resource (HR) system with copies of employee custom object records from Salesforce. You can synchronize the employee records in the HR system by receiving data change events from Salesforce. After receiving the events, you can process the corresponding insert, update, delete, or undelete operations in the HR system. Because the changes are received in near real time, the data in your HR system stays up to date.

Note - Available in: Performance, Unlimited, Enterprise, and Developer Editions. This is a pilot program and Change Data Capture isn’t generally available unless or until Salesforce announces its general availability in documentation or in press releases or public statements. Make your purchase decisions only on the basis of generally available products and features.

Event Subscription -

To subscribe to data change events, use CometD the same way you subscribe to other streaming events, like PushTopic events or platform events.

Subscription Channels -

a) The channel to subscribe to all data change events is: /data/ChangeEvents
b) The channel to subscribe to data change events for a standard object is: /data/AccountChangeEvent (for Account object)
c) The channel to subscribe to data change events for a custom object is: /data/Employee__ChangeEvent (for Employee__c object)

Example: This data change event is sent when an account is created.

{
"data": {
"schema": "IeRuaY6cbI_HsV8Rv1Mc5g",
"payload": {
"ChangeEventHeader": {
"entityName": "Account",
"recordIds": [
"001R0000002aV0B"
],
"changeType": "CREATE",
"changeOrigin": "com.salesforce.core",
"transactionKey": "001b7375-0086-250e-e6ca-b99bc3a8b69f",
"sequenceNumber": 1,
"isTransactionEnd": true,
"commitTimestamp": 1501010206653,
"commitNumber": 92847272780,
"commitUser": "<User_ID>"
},

"Name": "Acme",
"Description": "Everyone is talking about the cloud. But what does it mean?",
"OwnerId": "<Owner_ID>",
"CreatedDate": "2017-07-25T19:16:44Z",
"CreatedById": "<User_ID>",
"LastModifiedDate": "2017-07-25T19:16:44Z",
"LastModifiedById": "<User_ID>"
},

"event": {
"replayId": 6
}

},
"channel": "/data/ChangeEvents"
}

For getting the list of supported standard objects and Salesforce release note on this, visit the below link -
https://releasenotes.docs.salesforce.com/en-us/winter18/release-notes/rn_api_data_change_events.htm