Intro to Error Management 2.0

Heads up! Features added with the September launch of integrator.io v7.9.1 are marked NEW, below, for the convenience of customers who’ve already upgraded.

Error Management 2.0 offers a new paradigm for handling integration error tasks more efficiently and effectively. EM2.0 is being rolled out starting from mid-Sept. 2021 till mid-Oct 2021. If your account is not on EM2.0 yet, then you’ll receive email notification when your account will be migrated to the new error handling. Alternatively, you can migrate your account ahead of the rollout schedule by clicking the “Get Started now!” button.

Get started now!
[integrator.io customers] Get started now!
[eu.integrator.io customers]

• Prior to the release of integrator.io v7.9.1, Celigo strongly recommends that you perform this upgrade after business hours when no flows are running.
• NEW: Following the release of integrator.io v7.9.1, you may upgrade even while flows are currently running.

What’s been enhanced?

• Account dashboard
• Error notification from email alerts
• Flow run console
• Tools for searching and resolving errors in bulk
• New graphs to view key metrics
• Visibility to errors programmatically
• Automatically resolve duplicate errors

The new error management features are all designed to speed up your review process (compare to current features). And we promise much more to come to resolve errors faster and more intelligently!

NEW: Account dashboard

You can now see all in-flight flows and the results of completed flows in your account.

Account dashboard showing flows in progress

Account dashboard showing results of completed flows

In addition to seeing the summary of results from the runs, you can also click into the number of runs to see a flow’s run history within the selected date range and view individual detailed results.

Run history for a flow

From the Completed flows tab, you can also get to the errors for a flow by clicking into the count of open errors. Next, clicking into the error count for an integration step will in turn take you to the error list which is your new error-handling tool to search and resolve errors.

View all open errors for a flow

Error notification from email alerts

The platform checks for new and newly resolved errors every 15 minutes, then sensd any necessary email notifications with a summary of new, resolved, and open errors, along with error details.

Email notification

An account owner can subscribe to email error alerts within Flow Builder, as well as sign up other users to receive the notifications. (All email notifications set up in the previous error management are automatically applied to EM2.0, as well.)

Flow run console and run history

You now have a flow run console where you can watch a flow while it is running and access the errors as they’re reported (even before the flow run has completed).

The flow Run history tab displays run results for the past 30 days or the last 1,000 runs. It is important to keep in mind that the run history contains a snapshot of the flow’s results and will not be updated. For example, if you ran a flow a month ago with 100 errors and resolved 100 errors, the run history still reflects that that run produced 100 errors.

Tools for searching and resolving errors in bulk

When you are reviewing errors for a flow, you now can see all open errors and resolved errors for an integration step, filtered by the error source or time occurred. You can even perform a full-text search for specific error messages.

NEW: The error now includes integrator.io error classification to provide additional context about the error. In the future, the platform will try to run tailored recovery procedures for an error classification.

Within the error table, you can perform mass actions on the records, and you are no longer constrained by reviewing the errors per execution run. You can then retry any errors (either open or resolved error) and see whether the retry has completed. You can retry resolved errors as long as the resolved error has a timestamp (the date the error was created) within the past 30 days.

Retrying error is in progress

Retrying error has completed

When the retry is not successful and the error record has a trace key, you’ll see a new error with a label “Retry failed” added to its error message.

Retrying error was not successful

The retry status is maintained only for your current session. Whenever the session times out or you sign out, the “Retry failed” labels are refreshed.

New graphs to view key metrics

View your flow’s errors, successes, ignores, and average processing time/success record over time in new visual graphs, enabling you to spot patterns or trends that help you derive actionable information.

View the graphs for a single flow’s results in its dashboard console, and view the graphs for a single flow step’s results (such as import errors) in Flow Builder. 

Graphs summarize the results based on the timespan you select, from the following options:

• Results within a minute interval, when analyzing flow runs within the last 4 hours
• Results within an hourly interval, when analyzing flow runs between 4 hours and 4 days
• Results within a daily time interval, when analyzing flow runs over more than 4 days

Note: Although you can view the flow statistics for the past year, the error retry data is available only for the past 30 days.

Visibility to errors programmatically

With EM2.0, you can access your flows’ errors using the integrator.io API and then take action programmatically. Full access tokens are required for authentication. 

https://api.integrator.io/v1/flows/<_flowid>/<_exportId || _importId>/errors
GET

https://api.integrator.io/v1/flows/<_flowid>/<_importId>/resolved
PUT
Request body: { "errors": ["<_errorId1>", ... , "<_errorIdn>"]}

https://api.integrator.io/v1/flows/<_flowid>/<_importId>/retry
POST
Request body: { "retryDataKeys": ["<_retryDataKey1>", ... , "<_retryDataKeyn>"]}

You can add the following query parameters to this API to narrow down the results you want to see:

• occurredAt_lte (time value is UTC in ISO format)
• occurredAt_gte (time value is UTC in ISO format)
• Source

The API calls handle up to 1,000 errors; use pagination to fetch the next 1,000. 

For example, you could use the API within a routine or integration that displayed errors for a specific flow on Slack. Then, team members in Slack could mark these errors as resolved or retry them.

Automatically resolve duplicate errors

After you migrate your account to EM2.0, Auto-resolve will be enabled on the account so that you can leverage auto-resolve to mark duplicate errors resolved in the following scenarios:

• Auto-resolve will automatically mark all previous error records with the same trace key as the latest error record as resolved, leaving only the most recent errors open.
• If a record is successfully processed, then all errors that have the same trace key as the successful record’s are automatically marked as resolved – assuming that you fixed the flow configuration and reran it.

The trace key is the unique identifier of a record, as determined by integrator.io. For example, when exporting a record from NetSuite, the field "id" is automatically populated as the trace key. Then, Auto-resolve will apply AI/ML analysis to the trace key to attempt to resolve similar errors in identical records.

When you review resolved errors, you know which errors integrator.io auto-resolves by looking at the Resolved by column.

Errors auto-resolved by integrator.io

The Auto-resolve functionality relies on the trace key being configured correctly. You can override the default trace key with your own definition.

NEW: You can enable or disable Auto-resolve on a flow setting.

Enable/disable Auto-resolve for a flow

A more powerful way
to manage integration errors

Get started now!

Current error management vs. EM2.0

The following table gives you an at-a-glance comparison of concepts and features that will be deprecated in EM2.0 and how you will gather the information and resolve errors differently going forward:

Comments

Leave a comment below, and we’ll respond to your questions or suggestions.