The Celigo platform includes a set of advanced features designed to help you analyze integration flow runs and manage the system exceptions that can occur. The error management features include dashboards to help you quickly understand how your integrations are performing, as well as tools to help you find and resolve errors.
Contents
- Video walkthroughs
- View details for running and completed flows
- View a running flow and the run history
- Handle errors
- Receive error notification emails
- Analyze flow metrics
- Access errors via API
Video walkthroughs
Using dashboards
Investigating errors for a flow
View details for running and completed flows
You can see information about running flows and the results of completed flows in two locations:
- Account Dashboard: Click Dashboard on the left navigation menu to see details for all flows. You can filter by integration or other criteria as needed.
- Integration Dashboard: Within an integration, click the Dashboard tab to see the same information specific to that integration only.
Dashboard – Running flows tab showing flows in progress
Dashboard – Completed flows tab showing results
(While the run statistics are calculated for the selected Completed date range,
the Open errors count always contains the historical total for a completed flow.)
Tip: You can select an option under My Account settings to specify relative time display across all dashboards and resource tables instead of the absolute time, which is the default. (For example, to see that a run completed “4 hours ago” instead of 2022-02-10 3:30:02 PM, enable this option.) This system-wide setting can be set separately for each user. See Manage your profile for more information.
View a running flow and the run history
Run console: You can watch a flow as it's running and access the errors as they are reported (even before the flow run has completed) using the Run console. From within an integration, click a specific flow. The top half of the page shows the flow configuration, and the Run console appears on the first tab below. You can see detailed information for each step in the flow as it runs.
Note: The Run console reflects the status of the most recent run or the run currently in progress. If you begin to resolve errors, this is NOT reflected in the error counts.
Run history: For detailed information on past runs, use the Run history. It appears as a tab beneath the flow configuration when you're viewing a specific flow. You can also access it anytime from the Account Dashboard or Integration Dashboard when you are viewing Completed flows. Simply click the number in the Runs column for a specific flow to view its Run history, and expand the rows as needed to view details for each step of the flow.
The Run history displays run results for the last 1,000 runs, or however long your data retention plan allows. It is important to keep in mind that the Run history contains a snapshot of the flow’s results and will not be updated. For example, if you ran a flow a month ago with 100 errors and resolved 100 errors, the Run history still reflects that the run produced 100 errors.
In the Run console and Run history, you can view the color-coded status of flow runs. The status can be completed successfully, completed with errors, canceled, or failed. If the status of a flow run is Canceled, you can click the info icon next to it and view who canceled the flow.
Filtering the run history
Several options make it easier to focus on the history items you're interested in:
- Select range: Choose one of the set time ranges, such as Last 7 Days, or set a custom range to see only the runs that occurred during that period.
- Select status: Filter the history to see only runs that were completed, only runs containing errors, or other options.
- Hide empty runs: Exclude all jobs without records (i.e. Success, Ignored, and Errors columns are all 0).
Handle errors
Error counts are displayed in multiple places throughout integrator.io, such as on integration tiles and dashboards. Wherever you see a blue error count, click on it to drill in to those errors. You may see another page appear that breaks down the steps in the flow by application and shows the error count for each step. In that case, click on the error count for one of the steps to open the error list.
From here, you can mark errors as resolved or retry errors. You can also download the errors from the Actions (...) menu, as well as view the error details or edit the retry data.
Notice that Open errors, Resolved errors, and Retries appear on different tabs providing details to manage your errors and error retry runs.
After you mark an error as resolved, or while an error is being retried, it will only appear when you view Resolved errors.
Note: The error retry data is available for the most recent 20,000 errors for each flow step, or however long your data retention plan allows.
Search for and resolve/retry errors in bulk
When you are reviewing errors for a flow, you can see all open errors and resolved errors for an integration step, filtered by the error source, classification, or the time it occurred. You can even perform a full-text search for specific error messages.
The list includes the integrator.io error classification to provide additional context about the error. In the future, the platform will try to run tailored recovery procedures for an error classification.
Within the error list, you can perform mass actions on the records, and you are no longer constrained by reviewing the errors per execution run. You can then retry any errors (either open or resolved errors) and see whether the retry has completed. You can retry resolved errors as long as each resolved error has a timestamp (the date the error occurred) within the past 30 days or however long your data retention plan allows.
Retrying error is in progress
Retrying error has completedWhen the retry is not successful and the error record has a trace key, you’ll see a new error with a label “Retry failed” added to its error message.
Retrying error was not successfulThe retry status is maintained only for your current session. Whenever the session times out or you sign out, the “Retry failed” labels are refreshed.
Take the following class to learn more about this topic:
Automatically resolving duplicate errors
When you run a flow multiple times, you might get a long list of errors that includes duplicate errors. In this case, it's difficult to browse through the long error list with duplicate errors and troubleshoot effectively. Note that for troubleshooting, only the latest errors (that is, errors of the last run) matter, and you've got to prioritize only these errors. Auto-resolve removes duplicate errors and leaves you with only the open errors that require your attention, thus making troubleshooting easier and more effective.
Auto-resolve uses trace keys to find and resolve duplicate errors. The trace key is the unique identifier of a record, as determined by integrator.io. For example, when exporting a record from NetSuite, the field "id" is automatically populated as the trace key. Then, Auto-resolve will apply AI/ML analysis to the trace key to attempt to resolve similar errors in identical records.
Auto-resolve reduces duplicate errors through the following actions:
- All previous error records with the same trace key as the latest error record are marked as Auto-resolved, leaving only the most recent errors open.
- If a record is successfully processed after you fix a flow and rerun it, then all errors that have the same trace key as the successful record’s are automatically marked as Auto-resolved.
The following table explains flow scenarios where the Auto-resolve feature automatically takes care of duplicate errors (See also: this video walkthrough). Each scenario is independent; the scenarios are not linked to each other.
| Flow scenario | Without Auto-resolve | With Auto-resolve | Explanation for Auto-resolve |
|---|---|---|---|
| An open error exists for a record in a flow step. Now when the flow is run again, a new error occurs for the record with the same trace key. | Both, the new and previous errors display in the Open errors tab for the flow step. | Only the new error is shown in the Open errors tab. The previous error is auto-resolved and moved to the Resolved errors tab. | The latest error represents the most relevant blocker. The previous error is not relevant. |
| An open error exists for a record in a flow step. Now when the flow is run again, the record with the same trace key gets processed successfully because of fixes in the source data or flow setup. | The previous error will continue to be seen in the Open errors tab. | No error will be seen in the Open errors tab. The previous error is auto-resolved and moved to the Resolved errors tab. | Since the record is successfully processed, no further action is required. The previous error is not relevant. |
| An open error exists for a record in a flow step. Now, when the flow is run again, the record with the same trace key gets executed as an ignored record. | The previous error will continue to be seen in the Open errors tab. | No errors will be seen in the Open errors tab. The previous error is auto-resolved and moved to the Resolved errors tab. |
Since the record is executed as an ignored record, there will be no error. |
An Auto-resolved column on the Account Dashboard and Integration Dashboard indicates the number of errors the system resolved. When you review resolved errors in the error list, you can also see which errors integrator.io auto-resolved by looking at the Resolved by column.
Errors auto-resolved by integrator.io
Note: Auto-resolve does NOT update the error counts as errors are resolved on the Run console or the Run history as these reflect a snapshot of what occurred during the actual run. You will see the error count decrease in all locations that display overall flow statistics, such as on the Flows tab within the integration.
Configuring Auto-resolve
While Auto-resolve is enabled by default for each flow, you can disable or enable it for a specific flow in the flow settings.
Enable/disable Auto-resolve for a flow
The Auto-resolve functionality relies on the trace key being configured correctly. If needed, you can override the default trace key with your own definition.
Automatically retry processing for intermittent errors
When integrator.io classifies an error as Intermittent, up to four auto-retry attempts occur if Auto-resolve is enabled for the flow. The following table shows the auto-retry intervals for each attempt:
| Lapsed time | Auto-retry  |
| 00:00 | Error occurs and is classified as Intermittent –> auto-retry mode initiated |
| 00:30 | First attempt |
| 01:30 | Second attempt |
| 03:30 | Third attempt |
| 07:30 | Final attempt |
In the error list Classification column, you can see Intermittent during this time period. Hover over the icon to see when the next retry attempt will occur.
Receive error notification emails
Integrator.io checks for new and newly resolved errors every 15 minutes, then sends any necessary email notifications with a summary of new, resolved, and open errors, along with error details.
An account owner can subscribe to email error alerts within Flow Builder, as well as sign up other users to receive the notifications.
Note: If you have integrations in both sandbox and production environments, you must subscribe to notifications separately in each environment. When an error notification is sent from the sandbox, then "sandbox" is mentioned in the subject line.
Analyze flow metrics
Use the Analytics tab within an integration to view errors, successes, ignores, and average processing time per successful record over time in visual charts. This can help you spot patterns or trends, and take action accordingly.
Use the selection button on the top right to select up to eight data streams to include in the charts (for example, one integration-level data stream, and seven individual flows' data streams).
The charts summarize the results of flow runs that occurred in the timespan you select:
- Last 4 hours or less: Results displayed within a minute interval
- 4 hours to 4 days: Results displayed within an hourly interval
- More than 4 days: Results displayed within a daily time interval
Note: Although you can view the flow statistics for the past year, the error retry data is available for each flow step only for however long your data retention plan allows, or the most recent 20,000 errors, whichever occurs first.
Access errors via API
With Error Management, you can access your flows’ errors using the integrator.io API and then take action programmatically. Full access tokens are required for authentication.
https://api.integrator.io/v1/flows/<_flowid>/<_exportId || _importId>/errors
GET
https://api.integrator.io/v1/flows/<_flowid>/<_importId>/resolved
PUT
Request body: { "errors": ["<_errorId1>", ... , "<_errorIdn>"]}
https://api.integrator.io/v1/flows/<_flowid>/<_importId>/retry
POST
Request body: { "retryDataKeys": ["<_retryDataKey1>", ... , "<_retryDataKeyn>"]}
When making a GET method call, you can add the following query parameters to this API to narrow down the results you want to see:
- occurredAt_lte (time value is UTC in ISO format)
- occurredAt_gte (time value is UTC in ISO format)
- Source
An /errors API call can return up to 1,000 error records; use pagination to fetch the next 1,000.
A template available in the Marketplace demonstrates error handling automation via the integrator.io errors REST API. The Error automation via integrator.io APIs template contains two example flows that use Error Management features to automate retrying or resolving errors that meet certain conditions. This template is most useful for integration scenarios where errors do not need further resolution other than to retry the record as-is or for known issues that can be automatically resolved.
For example, you could use the API within a routine or integration that displayed errors for a specific flow on Slack. Then, team members in Slack could mark these errors as resolved or retry them.
Comments
6 comments
We are rolling this new error management framework in phases. So, please leave a comment here if you are interested in migrating to the new error management sooner rather than later.
As a reminder, if your account has Integration App (IA) or IA license or you plan to use IA, you'll need to wait till Feb 2021 to uptake the new features.
Thanks
Hi,
My flow doesn't give any errors, it just returns empty fields. How to troubleshoot that?
-Anna
Anna Orlova
I verified that your integrator.io account is not on the new error management. However, please raise a support ticket so they can discuss with you to better understand what fields are empty and then assist you with your questions.
Thank you
I'd love to be able to jump right into a data flows' errors. With this new dashboard, its adds a click because it shows a breakdown of the steps. I guess "parent jobs" is no longer a concept. Top video [Using Dashboards], 29 second mark. Just something I'll have to get used to. Love the product!
Hey,
I was working on Automatic resolve error by using trace key. So it worked fine.
But when i need again to go back on previous stage like i don't want to resolve error so i turned it off from setting.
Actually the problem arises when process is still resolving error and not showing latest errors in error log.
So please check it, if there is any situation please response.
Anuj kumar Maurya
I suggest that you log a case with support outlining steps to reproduce your scenario, what the problem is and what your expected result is. When the Auto-resolve errors is turned off, then integrator.io will not auto-resolve the errors.
Thanks
Please sign in to leave a comment.