Integration flows transfer data from a source application to a destination application. If you have customer record data in one application that you want to transfer to another application, an integration flow allows you to export data from the source application and import it into the destination application. You can create a flow to sync your shipping data with your billing info, or your inventory data with your customer lists. You can automatically stream the data in real time using webhook listeners, set up a flow to run on a scheduled basis, or manually trigger a flow to run as needed.
Important
Locate the API documentation for each application. The API documentation provided by the applications you are integrating contains information you need to reference when setting up your connections and modifying your data.
You must have Manage permissions to create a flow. The integration-selection drawer only shows integrations where you have permission to create flows. See Manage account and integration permissions.
You must have at least one integration in your account before creating a flow. If you haven't created one yet, see Create your first integration.
The flow builder lets you create and customize a flow to transfer data from one platform to another. You can use the flow builder to export and combine data from multiple applications, and import data into multiple applications.
To create a flow:
Note
Standalone flows — flows previously created without an integration (identifiable by integrations/none/flows in the URL) — are no longer supported. All existing standalone flows have been automatically migrated to an integration called Standalone Flows. No action is required on your part.
Note for API users
The integration ID (_integrationId) is required when creating a flow via API. Requests that omit this parameter will fail. See your API reference documentation for details.
-
Access the flow builder using one of the following:
-
From the left navigation menu, select Build > Flows, then select + Create flow.
-
From the home page, select Create > Flow.
-
-
In the drawer that opens, enter a name and an optional description for your flow.
-
Select an existing integration or select Create new integration.
Warning
If no integrations are available in the drawer, the Create flow option is disabled. Contact your admin for access.
-
Optionally, select a flow group.
-
Select Next. The flow builder opens.
-
Select Add source and choose the application that you want to export data from. If your preferred application does not appear in the list, you can use a generic HTTP connection to build flows between unlisted applications. See Set up an HTTP connection.
-
Select one of the following options from the What would you like to do? drop-down menu:
-
Export records from source application — Export data from your source application to be imported into your destination application. If you have already created a connection, select it from the drop-down menu. Otherwise, select + to create a connection.
-
Transfer files out of source application — Choose this if the source application data you want to export is a BLOB file that does not need to be modified before sending it to the next step in your flow.
-
Listen for real-time data from source application — Choose this if you are using a webhook listener in your source application.
-
-
Select Add destination / lookup to choose the app that you want to import data into.
-
Select one of the following options from the What would you like to do? drop-down menu:
-
Import records into destination application — Choose this if your flow does not need additional steps to generate the data you want to import into your destination application. If you have already created a connection, select it from the drop-down menu. Otherwise, select + to create a connection.
-
Lookup additional records (per record) — Choose this if your flow has a lookup step to retrieve additional information that is unavailable from the initial source application export. For more information on lookups, see Create HTTP import mapping with static or dynamic lookup. If you have already created a connection, select it from the drop-down menu. Otherwise, select + to create a connection.
-
Once you have established connections to both source and destination (or lookup) applications in the flow builder, select the Actions overflow (...) on the source application step in the flow builder canvas to define your export options.
You can choose to set up a flow schedule, transform data from the source application, filter data from the source application, or add a hook to your source application.
-
Schedule override: If you have a flow that contains multiple source export flow steps, you can set an individual schedule for each export to override your general flow schedule. See Override flow schedule.
-
Transformation: You can remove fields or change field names. For more information on data transformation, see Apply transform rules.
-
Output filter: You can filter for data that meets a set of criteria that you define. For more information on filters, see Apply filters.
-
Pre save page hook: Invoke a specific function in your records by creating a hook. For more information on hooks, see Export hooks.
Once your source application data contains the export data you want to import into your destination application, you must map the fields from the source application to their destination application counterparts. Select the mapping icon to define import mapping.
For more information on mapping records, see Map source record fields to destination.
You can also select the Define options button (+) to add a hook to your destination application. For more information on hooks, see Export hooks.
The flow builder has some functionality that only appears in specific circumstances. The above content only describes the options available for the simplest flow (export - import), but if you have lookup steps or multiple destinations in your flow, the flow builder provides additional customization options.
The records being exported from a particular source are grouped into pages of data. Each page is moved through the flow as processing becomes available, thus the integration is running on multiple pages at a time. The default number of records per page is 20, but you can modify page size to fine-tune flows for optimal throughput. This does not apply to webhooks and listeners, where data is sent in real-time and messages are queued, then run individually through the integration.
If you have multiple exports, each export is run as a separate job on your dashboard, thus data being sent from one export will not touch the data from another export.
You can choose to run source exports in parallel or serial using the Export execution mode setting under flow settings. This option is applicable when a flow has multiple sources. By default, new flows run sources in parallel; existing flows continue to run them serially for compatibility. This setting affects both manual and scheduled runs but doesn't apply when executing a single source manually. You can also cancel all active flow runs from multiple sources using the Cancel flow run option.
A lookup is an export to fetch additional data to be added to the original data that's going through the data flow. For example, you could pull order information from Shopify, then call back into Shopify to get details for each customer for those orders, then this customer information would be added to the order information going through the flow. A lookup is sending another query to another application or database. These results are used to refine or enrich the data being sent from your source application.
Select the + button next to Destinations & Lookups to add a lookup step.
The Celigo platform has two types of lookups: Lookup additional files (per record) and Lookup additional records (per record).
This option is available only for applications that have APIs with BLOB export capabilities. Choose this if there are files associated with each record in a given resource that you want to include as a BLOB file to add to each record pushed to the destination application. For more information on this option, see Example file transfer: Salesforce to NetSuite.
Choose this option if you want to supplement the data from your source application with additional data from another resource or location. The information retrieved by the lookup will be added to each record that passes through the lookup step.
Enter a name and description for the lookup step.
One to many: Select Yes (advanced) if the lookup records contain child records that should each be used as the main record. The Path to many field displays for you to select or enter the path to the section of the record that should be split into data for multiple record population. If the records you retrieve with the lookup share a one-to-one relationship with the records they are supplementing (meaning each record in the lookup should supplement each record imported), select No.
After making your selections, continue to the What would you like to export? section.
API Name — Select the name of the API you are connecting to. Available options vary depending on the APIs supported by the service you are using as your lookup source.
Operation — Select the operation to perform on the selected API to retrieve your lookup data. Available options vary depending on the API selected.
Select Launch in the Configure search parameters section to further refine your search parameters. The settings available from the pop-up window will vary depending on the API you selected for the service you are using as the source for your lookup step.
After you have configured the above settings, you can configure the export type.
Export type — Select the appropriate type for the needs met by your lookup:
-
All: Exports all data.
-
Delta: Exports only data that has changed since the last time the flow ran.
-
Once: Exports only one record for testing purposes.
-
Limit: Exports a defined number of records.
You can add a transfer step to send records to an alternate API database during a flow. Transfers are similar to lookups, but instead of contributing data to steps further down the flow, a transfer just copies the records according to your preferences to a secondary location. For more information on transfers, see Example file transfer: Salesforce to NetSuite.
You can add multiple destinations to a single flow. The data is migrated to each destination in the order defined by the flow. For more information about multiple destination flows, see Mapping responses from multiple destination applications.