Error counts are displayed in multiple places throughout the Celigo platform (integrator.io). Wherever you see an error count, click to drill down to those errors. For instance, if you click the error count link in an integration flow, you will see another page with the list of steps in the flow by application and the error count for each step. Here, click on the error count for any step to open the error list. See also, View error counts/errors in integrations, flows, and flow steps.
- Errors page/list
Manual resolution of errors (user resolved)
- Resolve option
- Retry option
- Retry or resolve errors one at a time or as a batch
- Troubleshoot errors based on the source
- Troubleshoot errors based on the classification
- Troubleshoot errors based on the timestamp
- Use the Error management template to retry/resolve errors
- Automatic resolution of errors (auto resolved)
The Errors page/list provides information about errors in an integration flow step. The Errors page has the following tabs:
- The Open errors tab, which shows errors that you can review and fix
- The Resolved errors tab, which shows both user-resolved and auto-resolved errors
- The Retries tab, which shows details of your error retries
The Actions (...) menu, which is available in the upper-right corner or as a column in the table enables you to perform more operations.
- In the Open errors tab, you can download the errors.
- In the Resolved errors tab, you can perform actions, such as download errors, view the error details, view the request, view the response, edit the retry data, and the like.
- In the Retries tab, you can cancel a retry any time before the retry is completed.
Note: The error retry data is available for the most recent 20,000 errors for each flow step, or however long your data retention plan allows.
This article covers how you can manually retry/resolve errors as well as how the Celigo platform automatically resolves errors.
You can use either of these two options on the Errors page to fix errors:
- Resolve: Move open errors to resolved errors for a specific purpose.
- Retry: Make changes where required, and try again if user errors are fixed.
Retry and Resolve options
You can use these options to fix one error at a time or many errors as a batch.
- Flow step bubble
- Integration page
You can manually resolve errors for specific purposes, such as filtering out the noise if you have too many errors to focus on a select few in the list; but, this resolution just cleans up the errors list temporarily and is not an effective fix. The Resolve option is also useful if you’ve disabled the auto-resolve setting and have to manually resolve duplicate errors.
- Search for duplicate/repeated errors based on the message, then select and resolve them as a batch.
- If you’ve resolved an error, its count increments in the User-resolved errors in the dashboard.
If you have a hunch about the reason for an error, you can Edit retry data and retry to test if it’s fixed. The following diagram illustrates how to identify and fix common errors. Errors with integrator.io as the source are often related to malformed handlebars expressions. External system errors are most frequently connection errors, permissions issues, data issues, or errors unique to the external system.
Note: Now for system outage errors, the Celigo platform automatically retries errors, or you can retry these errors without making any changes after some time.
You can use different approaches to retry or resolve errors. You can retry/resolve one error at a time systematically – or select several errors as a batch quickly.
To retry or resolve an error,
- Click the error count in any of the following places. Then, follow the links to the Errors page.
- Home page
- Integration page
- Flows > Name/Errors
- Flow builder – Step/Run history/Run console
- Dashboard in the left navigation menu
- To retry/resolve one error at a time, select the row and review Error details that provide cues to fix an error,
- If no change is required, let’s say in the case of an intermittent error, click Retry & next.
- If you want to make changes and test, let’s say a missing error, then edit the field value in Edit retry data and click Save, retry & next.
- If you want to resolve, click Resolve & next.
- To retry/resolve many errors as a batch, check the box next to the errors and the Add to batch box at the bottom is automatically checked. Then at the top of the page, click <n> retriable errors in the Retry box or <n> selected errors in the Resolve box.
Important: If an error is fixed after you’ve edited the retry data, make sure to make the same changes in the flow step (export, import, or the like).
Note: If you think any error resolution would help other platform users, please share your solution in the Community. However, if you’re still stuck, you’re welcome to ask a question in the Community or raise a Support ticket to resolve the issue.
When you are reviewing errors, you can filter by the source, classification, and time of errors, or you can even perform a full-text search for specific error messages. Filters can help to narrow the focus of your investigation and help you to detect patterns, examine details more carefully, or retry/resolve these errors in bulk. You can also filter errors and add tags as required.
Tip: While troubleshooting, after viewing details on the Errors page, you could go a bit deeper and view audit logs and, if required, filter on similar fields to try and identify errors. You can view the audit logs by clicking the Audit log tab in an integration.
You can troubleshoot by filtering on error Source. Below is an excerpt of source fields that you can filter on – you can choose a source based on the number or priority of errors, your expertise with a source, or any other criterion.
You can check for any similarities or cues in the error messages within the filtered set that might be helpful for troubleshooting. Also, view information on these sources in the Build flows or Connect to anything sections in the Help Center.
You can troubleshoot errors by filtering on Classification and trying to resolve one error class at a time.
Classification values such as “Missing,” “Intermittent,” “Duplicate (records)” and “Governance” indicate that the error originates from external applications (outside integrator.io).
Correct the mapping (for duplicate records).
(Governance errors have been reclassified into rate limit and too large errors).
Correct the concurrency value.
Intermittent errors related to tripping connections are auto-retried.
Enter missing details in the source.
Correct data types or incompatible values.
You can troubleshoot errors by filtering the Timestamp to view only those errors that occurred during a specific timeframe. It might also help to review your audit logs to check if any changes were made at this time.
Troubleshoot errors based on timestamp
You can also sort your errors and organize them in a specific order based on the columns in the error list. Use the sorting options such as timestamp (default - descending order to show the most recent errors), classification, code, message, or source (sorted in alphabetical order A-Z) to view errors, detect patterns, and troubleshoot or label errors. Note that the sorting option is available only in the unified error view of the Open errors tab.
You can sort 1,000 errors in one batch. The sorting criterion applies across error batches. When you move from one batch to another, if you change the sorting criterion, then the latest criterion is applied across batches.
Options to sort errors
The sections above discuss how you can filter and troubleshoot errors. Note that you can also sort and troubleshoot errors in a similar manner.
Within the error list, you can not only review the errors per execution run, but you can also perform actions on the records, that is retry/resolve errors individually or as a batch. You can retry any errors and see whether the retry has been completed. You can retry open errors as long as each open error has a timestamp (the date the error occurred) within the past 30 days or however long your data retention plan allows.
Retrying errors in progress
Retrying errors has completed
When the retry is not successful and the error record has a trace key, you’ll see a new error with the label “Retry failed” added to its error message.
The retry status is maintained only for your current session. Whenever the session times out or you sign out, the “Retry failed” labels are refreshed. See also: Manage your error retries
You can install the error management template from the Marketplace to retry/resolve errors. See the video.
The Celigo platform has features that automatically resolve errors, providing the following benefits:
- More efficient flows with fewer errors that require manual intervention
- Reduced time and effort in troubleshooting
- Flexibility to be in charge of your integration flows by enabling/disabling these options
These features are enabled by default in new flows/connections.
- Auto-resolve errors with the matching trace key in the flow settings
- Auto-recover rate limit errors in the connection settings
These two settings are in different locations, and they are not dependent on each other. We will discuss both these settings in the following sections.
Note: When an error is resolved using either of these options, you’ll see the error as Auto-resolved in the account dashboard, integration dashboard, and Errors page. The Auto-resolved resolution status not only refers to errors automatically resolved when the Auto-resolve errors with the matching trace key setting is enabled but also to the errors that are automatically resolved when the Auto-recover rate limit errors is enabled.
The Auto-resolve errors with the matching trace key setting is enabled by default in the flow settings of an integration. This setting automatically resolves repeated errors (duplicates) for all steps in a flow, and it’s shown as “auto-resolved.” It also automatically retries intermittent errors. When the Celigo platform has fixed an intermittent error, such as when a connection is offline and is back online, then you can see this status applied to the error.
While Auto-resolve is enabled by default for each flow, you can disable it for a specific flow in the flow settings.
To change the auto-resolve setting:
- In Flow builder, click the Settings button () at the upper right.
- In Settings, set Auto-resolve errors with the matching trace key. The default setting is Yes.
Note: The Auto-resolve functionality relies on the trace key being configured correctly. If needed, you can override the default trace key with your own definition.
Auto-resolve uses trace keys to find and resolve duplicate errors. The trace key is the unique identifier of a record, as determined by integrator.io. For example, when exporting a record from NetSuite, the field "id" is automatically populated as the trace key. Then, Auto-resolve will apply AI/ML analysis to the trace key to attempt to resolve similar errors in identical records.
Auto-resolve reduces duplicate errors through the following actions:
- All previous error records with the same trace key as the latest error record are marked as Auto-resolved, leaving only the most recent errors open.
- If a record is successfully processed after you fix a flow and rerun it, then all errors that have the same trace key as the successful record’s are automatically marked as Auto-resolved.
Note: Auto-resolve does not update the error counts on the Run console or the Run history as errors are resolved, because those totals reflect a snapshot of what occurred during the actual run. You will see the error count decrease in all places that display overall flow statistics, such as on the dashboards and Flows tab within the integration.
When integrator.io classifies an error as Intermittent, up to four auto-retry attempts occur if Auto-resolve is enabled for the flow. The following table shows the auto-retry intervals for each attempt:
|Error occurs and is classified as Intermittent –> auto-retry mode initiated
In the error list Classification column, you can see Intermittent during this time period. Hover over the icon to see when the next retry attempt will occur. See also 2:25–2.45 on this video timeline.
The Celigo platform helps your connection to automatically recover from rate limit errors. The Auto-recover rate limit errors option is enabled with a default Target concurrency level when you create a new connection.
When your connection runs into rate limit errors during a flow run, this feature automatically lowers the concurrency level to 1 and retries the request with an approximate wait time (delay) that doubles after every retry, for instance, 1 min, 2 mins, 4 mins… upto 1024 mins. The Celigo platform retries and waits for the API request to be successfully received, then it intelligently increments the concurrency level and repeats the process until all the records are processed or till it reaches the target level specified. At any time if it runs into errors, it again starts with the concurrency level 1. If rate limit errors are still encountered for the same errored records, even after performing a delay of approximately 1024 mins with a concurrency level of 1, then the setting is disabled and you should manually enable the Auto-recover rate limit errors setting again. So the maximum duration for rate limit auto-recovery would be 2047 min (~34h 7min), which is the total of all delays.
Using this setting, the Celigo platform automatically and proactively adjusts the concurrency level to recover from rate limit errors. You can view these adjustments or changes in the audit logs.
- in exports
- in lookups or imports that have One to many configuration
Only if you have administer or manage permissions, you can enable/disable or modify the Auto-recover rate limit errors and Target concurrency level settings. If you have monitor permissions, you can only view the connection settings.
To enable this setting,
- Go to Connections in any of the following ways:
- At the account level: From the main menu, click Connections. Search for any connection to find it quickly.
- At the integration level: Click Home > Your integration > Connections.
- At the flow level: Click Home > Your integration > Integration flows> Your flow > Flow builder > Connections.
- Click any connection to edit the settings.
- Under the Advanced section, select the Auto-recover rate limit errors check box.
Note: If you’re borrowing concurrency from another connection, you won’t see the Auto-recover rate limit errors check box, and automatically recovering from rate limit errors depends on whether this setting is enabled in the connection from which you’re borrowing concurrency.
- A default value is automatically set for the Target concurrency level based on your endpoint. For example, FTP – 1 and HTTP – 25. Change this setting if required based on the rate limit policy of your endpoint.
Concurrency level is non-editable and shows the concurrency that the Celigo platform is currently using at runtime. You can view the concurrency level changes during the auto-recovery period.
When errors are auto-resolved in flows using this option, you can view them in the Resolved tab of the Errors page and the account/integration dashboard.
- If you disable Auto-recover rate limit errors during a flow run, the auto-recovery process is cancelled. The flow will run using the default or the value you defined in the Target concurrency level before the auto-recovery started. You can view any errors in the Open errors tab.
- If you change the value in Target concurrency level during a flow run, the flow will continue to run with the new value and rate limit errors will be retried accordingly.
- If you'd like to check the flow progress when Auto-recover rate limit errors is enabled, you can monitor the dashboards or check the flow analytics to see how your flows are running and tweak the settings if required.
Auto-recover rate limit errors.
New connections: Use
falsein the body of a POST API request.
Existing connections: Use
connections/<_id>and make a GET API request to get the connection details. Then in the request body, update
falseand make a PUT API request.
autoRecoverRateLimitErrors is set to
true, you must also set a numeric value for
targetConcurrencyLevel based on your endpoint. If you don't set it, the platform sets default values.
You can view the Auto-resolved error count for completed and running flows only on the dashboard.
From the dashboard, you can also get details on the auto-resolved errors by clicking the error count link (in Open errors) and following links to the Errors page. Then, click Resolved errors and review rows with Auto-resolved in the Resolved by column.