Heads up! This is a pre-release document, containing additional EM2.0 features available with the launch of integrator.io v7.9.1, scheduled for 21 September 2021. Any changes or additions to the current version, for your reference, are marked NEW, below.
Error Management 2.0 offers a new paradigm for handling integration error tasks more efficiently and effectively. EM2.0 is being rolled out starting from mid sept 2021 till mid Oct 2021. If your account is not on EM2.0 yet, then you’ll receive email notification when your account will be migrated to the new error handling. Alternatively, you can migrate your account ahead of the rollout schedule by clicking the “Get Started now!” button.
Get started now!
[integrator.io customers] Get started now!
[eu.integrator.io customers]
- Prior to the release of integrator.io v7.9.1, Celigo strongly recommends that you perform this upgrade after business hours when no flows are running.
- NEW: Following the release of integrator.io v7.9.1, you may upgrade even while flows are currently running.
- Only the account owner can upgrade to the new error management experience. Afterward, all other shared account profiles (Administrator, Manager, or Monitor role) will be able to use EM2.0.
- All integrations in the account will be migrated to the new error management.
- This is a one-time migration. You won’t be able to switch back to the current error management version.
- Prior to your migration to EM2.0, Auto-resolve ran to clear duplicate errors. You may see a lower count of existing open errors after the migration.
- Salesforce – NetSuite v2 Integration App customers must migrate to Salesforce – NetSuite IO to make full use of EM2.0 features. (Also see the course Salesforce – NetSuite v2 to IO Migration: Details & Insights at Celigo University.)
What’s been enhanced?
- Account dashboard
- Error notification from email alerts
- Flow run console
- Tools for searching and resolving errors in bulk
- New graphs to view key metrics
- Visibility to errors programmatically
- Automatically resolve duplicate errors
The new error management features are all designed to speed up your review process (compare to current features). And we promise much more to come to resolve errors faster and more intelligently!
NEW: Account dashboard
You can now see all in-flight flows and the results of completed flows in your account.
Account dashboard showing flows in progress
Account dashboard showing results of completed flows
In addition to seeing the summary of results from the runs, you can also click into the number of runs to see a flow’s run history within the selected date range and view individual detailed results.
Run history for a flowFrom the Completed flows tab, you can also get to the errors for a flow by clicking into the count of open errors. Next, clicking into the error count for an integration step will in turn take you to the error list which is your new error-handling tool to search and resolve errors.
View all open errors for a flow
Error notification from email alerts
The platform checks for new and newly resolved errors every 15 minutes, then sensd any necessary email notifications with a summary of new, resolved, and open errors, along with error details.
Email notificationAn account owner can subscribe to email error alerts within Flow Builder, as well as sign up other users to receive the notifications. (All email notifications set up in the previous error management are automatically applied to EM2.0, as well.)
Flow run console and run history
You now have a flow run console where you can watch a flow while it is running and access the errors as they’re reported (even before the flow run has completed).
The flow Run history tab displays run results for the past 30 days or the last 1,000 runs. It is important to keep in mind that the run history contains a snapshot of the flow’s results and will not be updated. For example, if you ran a flow a month ago with 100 errors and resolved 100 errors, the run history still reflects that that run produced 100 errors.
Tools for searching and resolving errors in bulk
When you are reviewing errors for a flow, you now can see all open errors and resolved errors for an integration step, filtered by the error source or time occurred. You can even perform a full-text search for specific error messages.
NEW: The error now includes integrator.io error classification to provide additional context about the error. In the future, the platform will try to run tailored recovery procedures for an error classification.
Within the error table, you can perform mass actions on the records, and you are no longer constrained by reviewing the errors per execution run. You can then retry any errors (either open or resolved error) and see whether the retry has completed. You can retry resolved errors as long as the resolved error has a timestamp (the date the error was created) within the past 30 days.
Retrying error is in progress
Retrying error has completedWhen the retry is not successful and the error record has a trace key, you’ll see a new error with a label “Retry failed” added to its error message.
Retrying error was not successfulThe retry status is maintained only for your current session. Whenever the session times out or you sign out, the “Retry failed” labels are refreshed.
New graphs to view key metrics
View your flow’s errors, successes, ignores, and average processing time/success record over time in new visual graphs, enabling you to spot patterns or trends that help you derive actionable information.
View the graphs for a single flow’s results in its dashboard console, and view the graphs for a single flow step’s results (such as import errors) in Flow Builder.
Graphs summarize the results based on the timespan you select, from the following options:
- Results within a minute interval, when analyzing flow runs within the last 4 hours
- Results within an hourly interval, when analyzing flow runs between 4 hours and 4 days
- Results within a daily time interval, when analyzing flow runs over more than 4 days
Note: Although you can view the flow statistics for the past year, the error retry data is available only for the past 30 days.
Visibility to errors programmatically
With EM2.0, you can access your flows’ errors using the integrator.io API and then take action programmatically. Full access tokens are required for authentication.
https://api.integrator.io/v1/flows/<_flowid>/<_exportId || _importId>/errors
GET
https://api.integrator.io/v1/flows/<_flowid>/<_importId>/resolved
PUT
Request body: { "errors": ["<_errorId1>", ... , "<_errorIdn>"]}
https://api.integrator.io/v1/flows/<_flowid>/<_importId>/retry
POST
Request body: { "retryDataKeys": ["<_retryDataKey1>", ... , "<_retryDataKeyn>"]}
You can add the following query parameters to this API to narrow down the results you want to see:
- occurredAt_lte (time value is UTC in ISO format)
- occurredAt_gte (time value is UTC in ISO format)
- Source
The API calls handle up to 1,000 errors; use pagination to fetch the next 1,000.
Do you know that, after opting in to EM2.0, you can access your flows’ errors using the integrator.io API and then take action programmatically?
Check out this template that demonstrates error handling automation via the integrator.io error REST API. This template contains two example flows that use EM2.0 features to automate retrying or resolving errors that meet certain conditions. This template is most useful for integration scenarios where errors do not need further resolution other than to retry the record as-is or for known issues that can be automatically resolved.
For example, you could use the API within a routine or integration that displayed errors for a specific flow on Slack. Then, team members in Slack could mark these errors as resolved or retry them.
Automatically resolve duplicate errors
After you migrate your account to EM2.0, Auto-resolve will be enabled on the account so that you can leverage auto-resolve to mark duplicate errors resolved in the following scenarios:
- Auto-resolve will automatically mark all previous error records with the same trace key as the latest error record as resolved, leaving only the most recent errors open.
- If a record is successfully processed, then all errors that have the same trace key as the successful record’s are automatically marked as resolved – assuming that you fixed the flow configuration and reran it.
The trace key is the unique identifier of a record, as determined by integrator.io. For example, when exporting a record from NetSuite, the field "id" is automatically populated as the trace key. Then, Auto-resolve will apply AI/ML analysis to the trace key to attempt to resolve similar errors in identical records.
When you review resolved errors, you know which errors integrator.io auto-resolves by looking at the Resolved by column.
Errors auto-resolved by integrator.io
The Auto-resolve functionality relies on the trace key being configured correctly. You can override the default trace key with your own definition.
NEW: You can enable or disable Auto-resolve on a flow setting.
Enable/disable Auto-resolve for a flow
A more powerful way
to manage integration errors
Current error management vs. EM2.0
The following table gives you an at-a-glance comparison of concepts and features that will be deprecated in EM2.0 and how you will gather the information and resolve errors differently going forward:
| Error Management 1.0 | Error Management 2.0 |
|
|
|
|
|
|
Comments
Leave a comment below, and we’ll respond to your questions or suggestions.
Comments
1 comment
We are rolling this new error management framework in phases. So, please leave a comment here if you are interested in migrating to the new error management sooner rather than later.
As a reminder, if your account has Integration App (IA) or IA license or you plan to use IA, you'll need to wait till Feb 2021 to uptake the new features.
Thanks
Please sign in to leave a comment.