Intro to Error Management 2.0

Error Management 2.0 (EM 2.0) is a set of advanced features designed to help users manage the system exceptions that can occur when an integration flow runs. While some existing customers may be using the original error management features in integrator.io, all new customers' accounts are set up to use the enhanced features in EM 2.0 immediately. Many existing customers previously migrated to EM 2.0, and can take advantage of the latest enhancements immediately as well.

If you are unsure if your account is using EM 2.0, open any flow. If you see Run console and Run history tabs, your account has been upgraded.

Using Error Management 2.0 

Contents

• Video walkthroughs of EM 2.0 features
• View details for running and completed flows
• View a running flow and the run history
• Handle errors
  • Search for and resolve/retry errors in bulk
  • Automatically resolve duplicate errors
  • Automatically retry processing for intermittent errors
• Receive error notification emails
• Analyze flow metrics
• Access errors via API
• Compare EM 1.0 to EM 2.0

Video walkthroughs of EM 2.0 features

Using dashboards

Investigating errors for a flow

View details for running and completed flows

You can see information about running flows and the results of completed flows in two locations:

• Account Dashboard: Click Dashboard on the left navigation menu to see details for all flows. You can filter by integration or other criteria as needed.
• Integration Dashboard: Within an integration, click the Dashboard tab to see the same information specific to that integration only.

Dashboard – Running flows tab showing flows in progress

Dashboard – Completed flows tab showing results
(While the run statistics are calculated for the selected Completed date range,
the Open errors count always contains the historical total for a completed flow.)

Tip: You can select an option under My Account settings to specify relative time display across all dashboards and resource tables instead of the absolute time, which is the default. (For example, to see that a run completed “4 hours ago” instead of 2022-02-10 3:30:02 PM, enable this option.) This system-wide setting can be set separately for each user. See Manage your profile for more information.

View a running flow and the run history

You can watch a flow as it's running and access the errors as they are reported (even before the flow run has completed) using the Run console. From within an integration, click a specific flow. The top half of the page shows the flow configuration, and the Run console appears on the first tab below. You can see detailed information for each step in the flow as it runs.

Note: The Run console reflects the status of the most recent run or the run currently in progress. If you begin to resolve errors, this is NOT reflected in the error counts.

For detailed information on past runs, use the Run history. It appears as a tab beneath the flow configuration when you're viewing a specific flow. You can also access it anytime from the Account Dashboard or Integration Dashboard when you are viewing Completed flows. Simply click the number in the Runs column for a specific flow to view its Run history, and expand the rows as needed to view details for each step of the flow.

The Run history displays run results for the past 30 days or the last 1,000 runs. It is important to keep in mind that the Run history contains a snapshot of the flow’s results and will not be updated. For example, if you ran a flow a month ago with 100 errors and resolved 100 errors, the Run history still reflects that the run produced 100 errors.

Filtering the run history

Several options make it easier to focus on the history items you're interested in:

• Select range: Choose one of the set time ranges, such as Last 7 Days, or set a custom range to see only the runs that occurred during that period.
• Select status: Filter the history to see only runs that were completed, only runs containing errors, or other options.
• Hide empty runs: Exclude all jobs without records (i.e. Success, Ignored, and Errors columns are all 0).

Handle errors

Error counts are displayed in multiple places throughout integrator.io, such as on integration tiles and dashboards. Wherever you see a blue error count, click on it to drill in to those errors. You may see another page appear that breaks down the steps in the flow by application and shows the error count for each step. In that case, click on the error count for one of the steps to open the error list.

From here, you can mark errors as resolved or retry errors. You can also download the errors and access other options from the Actions menu, such as viewing the error details or editing the retry data. 

Notice that Open errors and Resolved errors appear on different tabs.

After you mark an error as resolved, or while an error is being retried, it will only appear when you view Resolved errors. 

Search for and resolve/retry errors in bulk

When you are reviewing errors for a flow, you can see all open errors and resolved errors for an integration step, filtered by the error source, classification, or the time it occurred. You can even perform a full-text search for specific error messages.

The list includes the integrator.io error classification to provide additional context about the error. In the future, the platform will try to run tailored recovery procedures for an error classification.

Within the error list, you can perform mass actions on the records, and you are no longer constrained by reviewing the errors per execution run. You can then retry any errors (either open or resolved errors) and see whether the retry has completed. You can retry resolved errors as long as each resolved error has a timestamp (the date the error occurred) within the past 30 days.

Retrying error is in progress

Retrying error has completed

When the retry is not successful and the error record has a trace key, you’ll see a new error with a label “Retry failed” added to its error message.

Retrying error was not successful

The retry status is maintained only for your current session. Whenever the session times out or you sign out, the “Retry failed” labels are refreshed.

Automatically resolving duplicate errors

Auto-resolve uses trace keys to find and resolve duplicate errors. The trace key is the unique identifier of a record, as determined by integrator.io. For example, when exporting a record from NetSuite, the field "id" is automatically populated as the trace key. Then, Auto-resolve will apply AI/ML analysis to the trace key to attempt to resolve similar errors in identical records.

Auto-resolve reduces duplicate errors through the following actions:

• All previous error records with the same trace key as the latest error record are marked as resolved, leaving only the most recent errors open.
• If a record is successfully processed after you fix a flow and rerun it, then all errors that have the same trace key as the successful record’s are automatically marked as resolved.

An Auto-resolved column on the Account Dashboard and Integration Dashboard indicates the number of errors the system resolved. When you review resolved errors in the error list, you can also see which errors integrator.io auto-resolved by looking at the Resolved by column.

Errors auto-resolved by integrator.io

Note: Auto-resolve does NOT update the error counts as errors are resolved on the Run console or the Run history as these reflect a snapshot of what occurred during the actual run. You will see the error count decrease in all locations that display overall flow statistics, such as on the Flows tab within the integration.

Configuring Auto-resolve

While Auto-resolve is enabled by default with EM 2.0 for each flow, you can disable or enable it for a specific flow in the flow settings.

Enable/disable Auto-resolve for a flow

The Auto-resolve functionality relies on the trace key being configured correctly. If needed, you can override the default trace key with your own definition.

Automatically retry processing for intermittent errors

When integrator.io classifies an error as Intermittent, up to four auto-retry attempts occur if Auto-resolve is enabled for the flow. The following table shows the auto-retry intervals for each attempt:

Lapsed time	Auto-retry
00:00	Error occurs and is classified as Intermittent –> auto-retry mode initiated
00:30	First attempt
01:30	Second attempt
03:30	Third attempt
07:30	Final attempt

In the error list Classification column, you can see Intermittent  during this time period. Hover over the icon to see when the next retry attempt will occur.

Receive error notification emails

Integrator.io checks for new and newly resolved errors every 15 minutes, then sends any necessary email notifications with a summary of new, resolved, and open errors, along with error details.

Email notification

An account owner can subscribe to email error alerts within Flow Builder, as well as sign up other users to receive the notifications. All email notifications previously set up in EM1.0 are automatically applied to EM2.0.

Analyze flow metrics

Use the Analytics tab within an integration to view errors, successes, ignores, and average processing time per successful record over time in visual charts. This can help you spot patterns or trends, and take action accordingly.

Use the selection button on the top right to select up to eight data streams to include in the charts (for example, one integration-level data stream, and seven individual flows' data streams). 

The charts summarize the results of flow runs that occurred in the timespan you select:

• Last 4 hours or less: Results displayed within a minute interval
• 4 hours to 4 days: Results displayed within an hourly interval
• More than 4 days: Results displayed within a daily time interval

Note: Although you can view the flow statistics for the past year, the error retry data is available for each flow step only for the past 30 days or the most recent 20,000 errors, whichever occurs first.

Access errors via API

With EM2.0, you can access your flows’ errors using the integrator.io API and then take action programmatically. Full access tokens are required for authentication. 

https://api.integrator.io/v1/flows/<_flowid>/<_exportId || _importId>/errors
GET

https://api.integrator.io/v1/flows/<_flowid>/<_importId>/resolved
PUT
Request body: { "errors": ["<_errorId1>", ... , "<_errorIdn>"]}

https://api.integrator.io/v1/flows/<_flowid>/<_importId>/retry
POST
Request body: { "retryDataKeys": ["<_retryDataKey1>", ... , "<_retryDataKeyn>"]}

When making a GET method call, you can add the following query parameters to this API to narrow down the results you want to see:

• occurredAt_lte (time value is UTC in ISO format)
• occurredAt_gte (time value is UTC in ISO format)
• Source

An /errors API call can return up to 1,000 error records; use pagination to fetch the next 1,000. 

For example, you could use the API within a routine or integration that displayed errors for a specific flow on Slack. Then, team members in Slack could mark these errors as resolved or retry them.

A more powerful way
to manage integration errors

Get started now!
[integrator.io customers] Get started now!
[eu.integrator.io customers]

Compare EM 1.0 to EM 2.0

The following table gives you an at-a-glance comparison of concepts and features that will be deprecated in EM2.0 and how you will gather the information and resolve errors differently going forward: