Listen for real-time changes from Microsoft SQL

Anitha Abraham

Updated January 19, 2026 08:02

Flow steps with Microsoft SQL as the source application can use Change Data Capture (CDC). It enables streaming data changes such as inserts, updates, and deletes to a Microsoft SQL server in near real-time and provides the following benefits:

Faster updates from the source to downstream systems, with fewer reconciliation issues.
Clearer handling of inserts, updates, deletes, and multi-row transactions, reducing conflicts.
Less maintenance since it avoids custom listeners or heavy polling, improving reliability and ease of use.

Create a Microsoft SQL listener

Tip

Before you get started, if you set up a connection to Microsoft SQL, creating your listener will be quicker and easier.

To create a Microsoft SQL listener and capture data changes in real time, configure the following settings:

1. Create a source – a new flow step

In the left navigation, go to Flow builder ((Build → Flow builder).
In Flow builder, click Add source and configure details in the new flow step.

2. Configure source application and step details

In Application, search and select the database connector Microsoft SQL from the list.
In Step type, select Listen for real-time data from source application.
In Name your step, be sure to provide a clear and distinguishable. Throughout the Celigo platform, you will have the option to choose this new listener and reuse it, and a unique identifier will prove helpful later when selecting it from among a list of listeners that you’ve created.
Optional: In Describe your step, Enter a clear and concise description of the flow step (resource) to help others understand its purpose. Keep this description updated as you make changes.

3. Configure general info for the listener

In Name your listener,The step name that you had earlier entered will be shown. Review and edit, if required.
Optional: In Description, the step description that you earlier entered will be shown. If you make any changes to your listener, ensure that you also update the description to reflect the current scope or details.
In Connection, select a Microsoft SQL connection from the list. If you don't have a connection, click Create connection at the end of the list and Set up a connection to Microsoft SQL.

4. Configure real-time export in listener

Tip

You can use regular expressions to specify patterns for databases, schemas, tables or columns.

In Tables, specify Microsoft SQL tables in a comma-separated list to monitor and capture events from them. For example, users, orders, products.

Tip

If you leave the Tables setting blank, events from all tables on the connection database are captured. Use table.exclude.list in Additional properties to ignore specific collections.
In Additional properties, enter key-value pairs to set the snapshot_mode, and exclude or include listening to specific tables, databases, schemas, or columns. You must always specify the full path for schemas, tables, and columns here.

For more details, see also FAQs – MongoDB listener for change data capture (CDC)
Required property
snapshot.mode is defined with a default value of no_data . If required, you can change the default value. Select when_needed or enter a valid value.

For more details on these values, see also FAQs – MongoDB listener for change data capture (CDC).
Optional properties
Based on your requirements, you can also specify the following optional properties:
- table.exclude.list – Specify any tables you don't want to listen to for changes. For example,
```
"table.exclude.list": "schemaName2.tableName1,schemaName2.tableName2"
```
- database.include.list – Specify any databases you want to listen to. For example,
```
"database.include.list": "dbName2,dbName3"
```
- database.exclude.list – Specify any databases you don't want to listen to for changes. For example,
```
"database.exclude.list": "databaseName2,databaseName3"
```
- schema.include.list –Specify any schemas you want to listen to and capture changes. For example,
```
"schema.include.list": "dbName2.schemaName1,dbName2.schemaName2"
```
- schema.exclude.list – Specify any schemas you don't want to listen to to for changes. For example,
```
"schema.exclude.list": "databaseName2.schemaName1,dbName3.schemaName2"
```
- column.exclude.list – Specify any column you don't want to listen to for changes. For example,
```
"column.exclude.list": "schemaName.tableName1.columnName1, schemaName.tableName1.columnName2"
```
- column.include.list – Specify any columns you want to listen to and capture changes. For example,
```
"column.include.list": "schemaName.tableName2.columnName1, schemaName.tableName2.columnName2"
```
In Fields to include, select which parts of the change event to include in the listener payload. By default, only the the after field is selected, but you can reduce payload size and simplify downstream processing by choosing only the fields you require (for example, include after and op fields to capture the updated record and operation type). Nested fields can also be included or excluded for more control.

5. Optional – Enter advanced listener settings

Enter any one or more of these advanced settings only if required for your scenario.
- In Max wait time:, set the maximum time (in seconds) for the listener to wait before sending a batch of events. Typically, events are sent when the Page size limit is reached. But, if the events are fewer, they are sent after the wait time. The default is 300 seconds. (Note that the min wait time is 30 seconds.)
- In Page size, specify how many records you want in each page of data. When a listener processes data in the context of a flow (where the data from the listener is sent right away to an import queue, the Celigo platform will break the data being exported into one or more smaller pages of records. The default system value (when you leave this field blank) is 250. There is no max value, but a page of data will automatically get capped when it exceeds 5 MB. Most of the time, the application that you are importing data into will bottleneck the page size value.
- In Data URI template, When your flow runs but has data errors, use this field to ensure that all the errors in your job dashboard have a link to the original data from the export application. This field uses a handlebars template to generate the dynamic links based on the data being exported.
- In Do not store retry data, check this box if you do NOT want the Celigo platform to store retry data for records that fail in your flow. Storing retry data can slow down your flow's overall performance if you are processing very large numbers of records that are failing.
- In Override trace key template, Define a trace key that the Celigo platform will use to identify a unique record. Any value you provide overrides the default trace key for your app. You can specify a single field, such as {{record.field1}}, or use a handlebars expression. For example, {{join "_" record.field1 record.field2}} generates a trace key such as 123_456. Note:If you have applied a transformation to exported data, reference its fields in the trace key template without the path record. – for example, {{field1}}.

After entering the required settings, click Save & close. (Enter any optional setting only if it is required for your scenario.)