A schema drift policy defines how a sync responds when the shape of your source data changes — for example, when a column is added, removed, or its data type changes. With this policy in place, Celigo sync automatically detects these changes and adjusts the destination schema so the sync keeps running without manual intervention.
A schema drift policy defines how your integration pipeline automatically handles changes (drift) in source data structures—such as when tables, columns, or data types are added, modified, or removed—without breaking downstream systems.
Key reasons to use a schema drift policy
- Prevent pipeline failures - Without rules, structural changes in your source system (like a dropped column or renamed field) can throw runtime errors and crash your data syncs.
- Automate maintenance - Instead of manually updating field mappings and target schemas whenever an upstream change happens, you can instruct the platform to automatically create new columns, ignore non-critical fields, or adjust data type sizes.
-
Protect downstream data integrity:
- You can safely accommodate additions (for example, auto-creating new destination columns when source fields are added).
- You can prevent accidental data loss (for example, stopping the system from dropping a column or truncating data when a field size is reduced).
- Granular control over changes - As shown in the following section on policy configuration, you can customize actions specifically based on what changed.
Set the default policy
Every source application has a sync-level schema drift policy that applies to all of its objects and exports by default.
When creating a sync (see Create a data ingestion sync), select Schema drift policy to review and modify the default settings for this source. This policy applies to every object or export you select in the following steps, unless you override it.
Override the policy for a specific object or export
When creating a sync (see Create a data ingestion sync), do one of the following, depending on your source application:
- Select Actions
next to an export, and then select Schema drift policy to add an export-level policy override.
- Select the Schema drift policy column to add an object-level policy override.
Configure the schema drift policy
Set specific schema drift policy rules depending on the type of change – choose what action to take when tables, objects, or fields are added, modified, or removed; specify behavior when field lengths change; and dictate how nested structures like objects or arrays are preserved or flattened.
When a table or object is added:
- Automate changes - Celigo sync automatically adds the table to the destination application.
- Ignore - The sync takes no action when a table is added. Not available for endpoints that only support exports.
When a column or field is added:
- Automate changes - Celigo sync automatically adds the column or field to the destination application.
- Ignore - The sync takes no action when a column or field is added to the source data.
When a column or field is removed or its size is reduced, Automate changes is the only available setting.
When a column or field's size is expanded, Automate changes is the only available setting. It handles data type or length increases detected in the source schema. For example, when Celigo sync identifies a Salesforce text area field, it sets the destination data type to VARCHAR(MAX) to accommodate longer historical data and avoid errors.
Schema structure: handling nested fields
Schema structure is a separate, field-level setting that controls how Celigo sync handles a nested object or array — distinct from the schema drift policy, which handles added, removed, or resized columns.
When a new field is an object:
-
Preserve - Import the JSON object (
{"key1": "value", "key2": ...}) as a single field value. - Flatten - Add the nested fields and values to the parent's table as additional columns, at the same level as the parent's other fields.
When a new field is an array:
-
Preserve: Import the JSON array (
["value1", "value2", ...]) as a single field value. - Normalize: Add the nested fields and values as a separate destination table, linked back to the parent.
Examples
A source column is removed
Your Salesforce source has a contacts object with a middle_name column, synced to a contacts table in Snowflake. An admin deletes middle_name from Salesforce.
On the next run, Celigo sync detects the removal and logs it as a schema drift event, but makes no change to the destination table — the middle_name column and its historical values remain in Snowflake exactly as they were. Your sync keeps running normally, and queries that reference middle_name continue to return the data loaded before the source column was deleted.
You can see the event in the sync's event log, filtered by the Schema drift type. See Interpret the event log.
A source column is added
The same contacts object gains a new preferred_language column in Salesforce.
What happens next depends on your policy. If Added columns are set to Automate changes, Celigo sync adds preferred_language to the destination contacts table on the next run and begins loading its values. If Added columns are set to Ignore, the sync takes no action and the new column never reaches Snowflake — useful when you want to control your destination schema deliberately rather than have it grow with the source.
Note the asymmetry between these two cases: additions are optional and configurable, while removals are always handled the same way and never affect data already at the destination.