Articles in this section

Create a data ingestion sync

Creating a Celigo sync involves three standard steps: configuring your source, setting up your destination, and scheduling the run. This article covers the entire wizard flow, from choosing a source connection and selecting objects or exports, through picking a destination connection, to setting a run schedule, so you can get a working sync end to end.

To learn more about data ingestion and Celigo sync, see Celigo sync overview.

Prerequisites:

All Celigo subscriptions include one active sync. Contact your account manager for details on Data Ingestion add-ons and volume packs.

What a sync contains

At a minimum, every sync includes the following:

  • Integration scope - Name and organize the sync within an integration that exclusively targets the same destination application.
  • Source application - Identify a single source application alongside its data (tables, records, objects, fields, and key characteristics).
  • Destination application - Select a single destination application with configured data ingestion settings (database, table, and schema).
  • Execution schedule - Set a run frequency (on demand or scheduled) using either a historical (initial) or periodic refresh data flow.

Create a sync

Configure your source data, select your target destination, name your schema, and set a run schedule.

  1. Go to Home home-icon.png, select Create, and then select Sync.
    create-sync.png
  2. Choose the integration that will hold the sync. Choose one with the same destination as your new sync, or select Create integration to create a new one (see Create or open an integration).

    Note

    An integration can only contain syncs that share the same destination application.

  3. Select Next to open the Create sync wizard.

Step 1 - Configure the source application

  1. Choose a source - Select the source application that will provide data for the sync. Choose from any prebuilt, database, or universal application you can connect to, in the cloud or on-premises.
  2. Connect to the source - Select a connection, or select Create connection to make a new one (see Create a connection to an application). You can change your connection later.
  3. Set the sync-level schema drift policy - A schema drift policy defines how a sync responds when the shape of your source data changes — for example, when a column is added, removed, or its data type changes. Select Schema drift policy to review and modify the default settings for this source. This policy applies to every object or export you select in the following steps, unless you override it (see also Configure a schema drift policy).
    schema-drift-policy.png
  4. Select objects or exports to sync. Preview the results for the data available through your connection and select an export, object, or saved search name to choose which fields to sync. For the full field-level configuration within this step — choosing which fields to sync, naming the destination table, setting a primary key, and more, see the following articles for the exact steps:

    Important

    When merging tables, every source object requires at least one primary key. If you normalize a child object (preserving the array), you must set a primary key at each normalized array level for its destination table. If the child object is flattened, a primary key can be assigned at any level of the hierarchy.

Step 2 - Configure the destination application

  1. Choose a destination - Select the destination that will ingest and store all source data from the sync. For destination-specific details — including a table-name prefix for NSAW and the full data type mapping and reserved-word reference for each destination — see the following:
  2. Connect to the destination - Choose an established connection, or select Create connection to create a new one (see Create a connection to an application).

    Caution

    Avoid sharing connections with flows scheduled at the same time, as running them simultaneously can reduce performance.

Step 3 - Define sync settings

Name and describe your sync and choose whether to receive email notifications (see Edit or view a sync's settings). Next, decide how often the sync should run (see Schedule and run a sync for the full set of scheduling options).

What happens next

After you save your sync's settings, the wizard is complete and you're returned to the sync's dashboard. The dashboard shows a "Ready to run" state — your sync is active, but hasn't loaded any data yet. Select Run to trigger the first run yourself, or wait for its next scheduled run.

Learn more