When querying an endpoint, in many cases you can further refine the data you’re retrieving:
-
All – always export all data
-
Delta – export only data created/updated since the last time the flow ran, based on a given date field
-
Once – export records only once by setting a flag in the source system as each record is processed
-
Limit – export between 1 and 100 records
You can also retrieve additional source data in a “lookup” flow step, following an export or import.
Tip
Each “page” of exported data cannot exceed 5 MB. (When integrator.io gets data from a source, it splits the data into pages to make the records more manageable for queueing and further processing. You can modify the default page size of 20 records in the Page size advanced setting of an export.) To reduce the size of the records within a page, you can modify the data in a preSavePage hook. That 5 MB limit also applies as the page moves “downstream” in a flow; be aware of how you might be increasing the size, for example, by adding fields in a lookup.
Known limitation: If an API does not offer functionality to support a delta export, you can achieve similar results by leveraging other integrator.io features, such as sending a value in the query string, creating and updating a custom field, or filtering records based on a known export date.
You can configure an export or lookup and run a preview to see sample JSON structure before running it in a flow. You can choose to preview up to 100 records.
HTTP connectors also display full response and request details, including any error messages.
For more information:
The most common and fundamental way to sync data is to retrieve a record from one system and insert or update it in another. The following field “mapping” methods are readily available for most apps:
-
Simple mapping – Prepare source records for their corresponding fields in the destination app
-
Hard-coded mapping – Specify a single numeric or string value for a destination field
-
Multifield/handlebars expressions – Perform complex low-code operations to customize and restructure fields for import
-
Lookups – Dynamic and static queries of the destination app
-
Lookup caches – Create in-memory lookups that can be easily repurposed throughout flows
-
-
Transformation mapping – Modify, restructure, or simplify data from the source system before it's processed in a flow
-
Data type – Select the value expected by the destination app, including any override you want to apply
-
Conditional mapping – Set a criteria, such as only when “updating a record”
-
Request body mapping – Alternatively, build a template with handlebars expressions to represent the request HTML body
-
Complex JSON mapping – Switch to Mapper 2.0 for complete visual control over data types and record structure in HTTP-based, EDI, and file transfer imports
For commonly used applications, the Celigo platform can automatically suggest destination field mappings that you have not yet added. Auto-map will add field mappings based on common configurations between two applications and how frequently these field mappings are successfully configured by integrator.io customers.
Tip
Each page of data processed in an import cannot exceed 5 MB. See Maximum record size for further implications of increasing data within a flow.
For more information:
Within a flow’s destinations, you can split the exported data into different branches for conditional processing. You can then import records of each branch into different destination applications or perform lookups. You can also add, modify, reorder, delete, and merge branches as required – filtered either with logical rules or JavaScript.
For more information:
You can debug a flow in test mode to ensure the flow processes your records as expected before enabling the flow. Enter mock data manually or make calls to your source and destination apps to retrieve live data to use for testing purposes.
When you run a flow in test mode, you can review the processing behavior of each flow step option (transformation, mapping, filter, or hook) and fine-tune your flow configuration to handle all variations of results or responses from each application endpoint.
For more information:
You can run any flow on demand or set it to initiate with a call from a webhook listener (as a “real-time” flow). You can also schedule flows initiated from an export or file download with the following options:
-
Preset times
-
cron expressions
-
Daisy-chained – Initiated after a selected flow
-
Export delta override – For NetSuite, Salesforce, HTTP, and REST exports
For more information:
Known limitation: Export delta overrides are not available for file providers such as Amazon S3 and Google Drive.
The Celigo platform allows you to clone a flow, distribute integrations as templates, package flows, and download to another account.
-
Make your integrations more portable by referencing resources with aliases
-
Test in a non-production environment before deploying into production
For more information:
Integration Lifecycle Management simplifies and accelerates the process of implementing, modifying, testing and deploying integrations. It supports revisions and enables you to:
-
Create snapshots as backups
-
Compare diffs between two integrations when merging changes
-
Pull changes between a remote and current integration
-
Revert an integration to a selected revision
-
Review changes in a revision
For more information:
Access integration resources via the RESTful API:
-
Get, update, test, delete, and debug connections
-
Get, create, update, delete, and clone flows, exports, and imports
-
Run an export, import, or flow on demand from your external application
-
Get, create, update, delete, and clone integrations
-
Get, resolve, and retry all errors for an import, as well as get all errors for an export
When invoking the integrator.io API, you must supply a secure, valid Bearer token retrieved from your account. Your account’s access allows for the creation and revocation of fine-grained API tokens, where a specific set of exports, imports, JavaScript APIs, or connections can be included in its scope. An audit log monitors the use and modification of tokens for recordkeeping.
For more information:
With AI as the driving force behind it, Celigo iPaaS enhances building and error management, bridges knowledge gaps between IT and business users, and provides insights learned from thousands of customers.
Democratizing the integration process ensures team alignment, prevents bottlenecks, and drives innovation amongst other benefits. One of the biggest challenges is the handoff between technical and non-technical teams.
Celigo AI allows non-technical users to better understand the integration process. Integration and coding aren’t always transparent, so AI acts as a translator. It helps clarify what the automations and configurations mean. Our chat-enabled editors that support natural language and auto-documentation of flows contribute to streamlining this process.
For more information: