Flows transfer data from a source application to a destination application. A flow exports data from the source and imports it into the destination, for example to sync shipping data with billing information, or inventory data with customer lists. A flow can stream data in real time through a webhook listener, run on a schedule, or run when you start it manually.
Use this article to create a flow in an integration; add sources, destinations, lookups, and other steps such as AI agents, tools, and guardrails; and find your flows on the Flows page.
Important
Locate the API documentation for each application. The API documentation provided by the applications you're integrating contains information you need to reference when setting up your connections and modifying your data.
You must have Manage permissions to create a flow. The integration-selection drawer only shows integrations where you have permission to create flows. See Manage account and integration permissions.
You must have at least one integration in your account before creating a flow. If you haven't created one yet, see Create your first integration.
The flow builder lets you create and customize a flow to transfer data from one platform to another. You can use the flow builder to export and combine data from multiple applications, and import data into multiple applications.
Note
Every flow belongs to an integration. Flows that were created without an integration are in the Standalone Flows integration.
Note for API users
The integration ID (_integrationId) is required when you create a flow through the API. Requests without it fail. See the integrator.io API reference for details.
To create a flow:
- Access the flow builder using one of the following:
- From the left navigation menu, select Build > Flows, then select + Create > Create flow.
- From the home page, select Create > Flow.
- In the drawer that opens, enter a name and an optional description for your flow.
-
Select an existing integration or select Create new integration.
Warning
If no integrations are available in the drawer, the Create flow option is disabled. Contact your administrator for access.
- Optionally, select a flow group.
- Select Next. The flow builder opens.
- Select Add source, and then select the application that you want to export data from. If your application isn't in the list, use a generic HTTP connection to build the flow. See Set up a custom HTTP connection.
- Select one of the following options in Step type. The options available depend on the application:
- Export records from source application — Export data from your source application to be imported into your destination application. If you have already created a connection, select it from the drop-down menu. Otherwise, select Create connection.
- Transfer files out of source application — Select this if the source application data you want to export is a BLOB file that doesn't need to be modified before sending it to the next step in your flow.
- Listen for real-time data from source application — Select this if you're using a webhook listener in your source application.
- On the flow builder canvas, select Add step. The Create destination / lookup drawer opens.
- In Application, select the application that you want to import data into or look up data from, and then select one of the following options in Step type:
- Import records into destination application — Select this if your flow doesn't need additional steps to generate the data you want to import into your destination application. If you have already created a connection, select it from the drop-down menu. Otherwise, select Create connection.
- Look up additional files (per record) — Select this to retrieve files associated with each record and add them as BLOB files to the records sent to the destination. This option appears only for applications with APIs that support BLOB exports.
- Look up additional records (per record) — Select this if your flow has a lookup step to retrieve additional information that is unavailable from the initial source application export. For more information on lookups, see Create HTTP import mapping with static or dynamic lookup. If you have already created a connection, select it from the drop-down menu. Otherwise, select Create connection.
- Transfer files into destination application — Select this to send files to your destination application without modifying them.
- In Name your step, enter a name for the step. Optionally, in Describe your step, enter a short description of what the step does so others understand its purpose. The description can be up to 300 characters.
After you connect your source and destination (or lookup) applications in the flow builder, select Actions overflow (...) on the source step to define your export options.
You can set up a flow schedule, transform data from the source application, filter data from the source application, or add a hook to your source application.
- Schedule override — If your flow has multiple source exports, you can set a schedule for each export that overrides the flow schedule. See Override flow schedule.
- Transformation — You can remove fields or change field names. For more information on data transformation, see Apply transform rules.
- Output filter — You can filter for data that meets a set of criteria that you define. For more information on filters, see Apply filters.
- Pre save page hook — Invoke a specific function in your records by creating a hook. For more information on hooks, see Export hooks.
After your source application data contains the export data you want to import into your destination application, you must map the fields from the source application to their destination application counterparts. Select the mapping icon to define import mapping.
For more information on mapping records, see Map source record fields to destination.
You can also select Define options (+) to add a hook to your destination application. For more information, see Import hooks.
Lookup steps and multiple destinations add customization options to the flow builder beyond a basic export-to-import flow. Some of these options appear only in specific circumstances.
The records being exported from a particular source are grouped into pages of data. Each page moves through the flow as processing becomes available, so the integration processes multiple pages at a time. The default number of records per page is 20, but you can modify page size to fine-tune flows for optimal throughput. This doesn't apply to webhooks and listeners, where data is sent in real-time and messages are queued, then run individually through the integration.
If you have multiple exports, each export runs as a separate job on your dashboard, so data from one export doesn't affect data from another export.
You can run source exports in parallel or in serial with the Export execution mode setting in flow settings. This option is applicable when a flow has multiple sources. By default, new flows run sources in parallel; existing flows continue to run them serially for compatibility. This setting affects both manual and scheduled runs but doesn't apply when executing a single source manually. You can also cancel all active flow runs from multiple sources using the Cancel flow run option.
A lookup is an export to fetch additional data to be added to the original data that's going through the data flow. For example, you could pull order information from Shopify, then call back into Shopify to get details for each customer for those orders, then this customer information would be added to the order information going through the flow. A lookup is sending another query to another application or database. These results are used to refine or enrich the data being sent from your source application.
To add a lookup step, select Add step on the flow builder canvas, or select + between two steps.
The Celigo platform has two types of lookups: Look up additional files (per record) and Look up additional records (per record).
This option is available only for applications that have APIs with BLOB export capabilities. Select this if there are files associated with each record in a given resource that you want to include as a BLOB file to add to each record pushed to the destination application. For more information on this option, see Example file transfer: Salesforce to NetSuite.
Select this option if you want to supplement the data from your source application with additional data from another resource or location. The lookup adds the information it retrieves to each record that passes through the lookup step.
Note
A lookup adds data to each record, so a page that was within the 5 MB page size limit at export can go past 5 MB after the lookup. When this happens, integrator.io automatically splits the page between records and the flow continues. You don't need to change your page size settings. The flow still fails if a single record is larger than 5 MB.
Enter a name and description for the lookup step.
One to many: Select Yes (advanced) if the lookup records contain child records that should each be used as the main record. The Path to many field displays for you to select or enter the path to the section of the record that should be split into data for multiple record population. If the records you retrieve with the lookup share a one-to-one relationship with the records they are supplementing (meaning each record in the lookup should supplement each record imported), select No.
After making your selections, continue to the What would you like to export? section.
API Name — Select the name of the API you're connecting to. Available options vary depending on the APIs supported by the service you're using as your lookup source.
Operation — Select the operation to perform on the selected API to retrieve your lookup data. Available options vary depending on the API selected.
Select Launch in the Configure search parameters section to further refine your search parameters. The settings in the window vary depending on the API you selected for the service you're using as the source for your lookup step.
After you set API Name, Operation, and search parameters, configure the export type.
Export type — Select the appropriate type for the needs met by your lookup:
- All — Exports all data.
- Delta — Exports only data that has changed since the last time the flow ran.
- Once — Exports only one record for testing purposes.
- Limit — Exports a defined number of records.
You can add a transfer step to send records to an alternate API database during a flow. Transfers are similar to lookups, but a transfer copies the records to a secondary location instead of adding data to later steps. For more information on transfers, see Example file transfer: Salesforce to NetSuite.
You can add multiple destinations to a single flow. The data is migrated to each destination in the order defined by the flow. For more information about multiple destination flows, see Mapping responses from multiple destination applications.
Find flows on the Flows page
The Flows page (Build > Flows) lists the flows in every integration you have access to.
By default, the list is sorted by Last run, with the most recent first. To sort by a different column, select its header. Select the header again to reverse the order. You can sort by Name, Last run, and Last updated. You can't sort by Integration, Status, Schedule, or Actions.
Sorting works together with the name search, the Integration filter, and the Status filter, so you can narrow the list and then sort the results.
Tip: To find flows that haven't run recently, sort by Last run with the oldest first.
Flows that have never run appear at the end of the list, whichever direction you sort by Last run.
For a multi-instance base flow, Last run shows the most recent run of any of its instances. See Manage your multi-instance flows.