Handle errors in integrator.io (EM 1.0)

Note: This article applies to Error Management 1.0. If your account has been upgraded to Error Management 2.0, see Intro to Error Management 2.0. To determine if your account has been upgraded, open any flow. If you see Run console and Run history tabs, your account has been upgraded.

integrator.io provides error messaging in a couple formats for system exceptions that occur when a flow, connection, or data transfer fails:

• Configure error notifications to alert people in your organization as needed
• View errors in the dashboard to understand the steps needed to resolve problems

You can view success and error counts in the Dashboard tab. The counts are updated as the flow progresses.

1. Click the integration tile from the Home page.
2. Click the Dashboard tab.
3. Hover over the red error numbers and click the View link in the Error column.

From here, you can download the errors to a spreadsheet (in CSV format, make the necessary changes, and upload the file by clicking Upload processed errors.

Note: Any unresolved errors in the spreadsheet are likely to create a duplicate error when uploaded.

Contents

• Integration dashboard
• Status colors and their meanings
• Identify error source
• Error categories
  • Failure during initial data export
  • Errors during lookup
  • Errors during import
  • Errors during webhook exports
• Review errors
• Error notification
• Manage and handle errors
  • Resolve or retry selected errors
  • Resolve or retry errors in bulk

Integration dashboard

The integration dashboard is the primary entry point for managing errors:

• Retry, resolve all or a selected job records in case of an error
• Filter jobs based on a given flow
• View, troubleshoot and manage job errors
• Filter jobs based on their status, date, and errors
• Modify data flow configuration and run a data flow using the Actions overflow (...) menu

• Success – Number of records successfully imported or looked up at each connector. In the collapsed view, it shows the sum of successes at each stage.
• Ignored - The number of records that the system skipped from being imported. These are records that are not counted under successes or errors, but were skipped while processing the import step.
• Errors – This column indicates the following:
  • Number of errors that occurred at each connection while running the data flow.
  • If the whole page failed to import due to an error returned by the endpoint, the dashboard displays a single error for the whole page. If you were exporting 10 pages and each page errored out, then this column displays 10 separate errors (one for each page). When you retry errors, you can either retry all or retry only the selected errors. Once retried, the error is removed and the success count is incremented to match the successfully imported record count.

Note: If you are unable to see retry data for an error, the record was successfully imported, but the error occurred on a mapped field, and only that field was excluded from the import (while other fields on the record imported successfully). Since the record was successfully imported, the retry data is not available. Retry data is only available for records that fail to be imported.

• • It is also possible that only a few records of your page had errors while others were imported successfully. In such cases, one error per each errored record is counted under the corresponding connector.
  • Depending on your configurations and the endpoint, you might get multiple errors even for a single record. As an example, if your endpoint returns two separate errors because the record being imported was missing both the phone number and the email field, it could be counted as two errors under the corresponding connection. If you have multiple errors for a single record, you will have one record for both errors since both errors were caused by a single record. If you fixed the error and retried the import, then both of those errors are removed and the import counts as a single success.
• Resolved – The number of errors that you marked as resolved. When an error occurs in your data flow, you can fix the error and retry the record, but if you don't want to do that (or the error was fixed outside of integrator.io), then you can mark the error as resolved so that they don't appear in your dashboard. Manually editing a record with an error to mark it as resolved will not affect future records processed by your flows. If your flow encounters other records with the same error, you will have to manually set each record that receives the error as resolved.
• Pages – Total number of pages exported by your data flow. If your page size was 50 and your flow exported 500 records, then the Pages column will count 10 pages.
• Duration – Time taken by your data flow to complete its operation. Once expanded, it shows the duration taken at each connection.

Note: For FTP file integrations, the Actions icon on the dashboard displays a download link on the job record to view files that were exported (downloaded) from an FTP host. These files are kept on record for 30 days. You can use this feature for record keeping and error recovery.

Status colors and their meanings

Status	Meaning
	Each step in your data flow completed successfully. You can see the stats for successfully imported records under each column.
	The execution of your flow failed. See the Errors column for failure reasons.
	The execution of your flow was canceled. It could have been canceled by someone through the dashboard or automatically canceled by integrator.io if an unexpected error condition occurred while executing your flow.
	The execution of your flow was completed with both successful and unsuccessful imports or lookups. You can review the stats about which got completed successfully and which had errors by expanding the row in the dashboard.
	The execution of your flow was completed, but with errors and without any success.
	The execution of your flow was completed, but with errors, resolved errors, and successes. The blue color means that the job included some errors that were marked as resolved via the dashboard.
	The execution of your flow was completed only with errors marked as resolved via the dashboard.

Identify error source

The following diagram illustrates how to identify and fix common errors. integrator.io errors are often related to malformed handlebars expressions. External system errors are most frequently connection errors, permissions issues, data issues, or errors unique to the external system.

Error categories

Flows can generate errors at different stages of execution, categorized as follows:

Failure during initial data export

In this case, the status of the job is marked as failed with the reason for the failure. You can't retry these errors because the flow execution didn't start properly. Instead, the flow will be executed automatically at its next scheduled time or you can run the flow again manually. Such errors include those that occur while running an export post processor (such as transformation rules, filters, or hooks).

Errors during lookup

Errors while executing a lookup can occur when running pre or post processors (such as filters, transformations, mappings, or hooks) fall into this category. The execution is marked as completed but with errors, and you can retry these errors later. These errors could result for all the records of a page or only for specific records. If the whole page had errors, you will see an array of records under retry data, but if each individual record had errors, you will see a single record under retry data. To retry an error, you might need to edit the retry data depending on the type of the error.

Errors during import

This includes any error that occurred while executing an import, as well as when running pre or post processors (such as filters, transformations, mappings, hooks, or any file uploading). The execution is marked as completed, but with errors on it, which can be retried later. These errors could result for all the records of a page or only for specific records. If the whole page had errors, you will see an array of records under retry data, but if each individual record had errors, you will see a single record under retry data. To retry an error, you might need to edit the retry data depending on the type of the error.

Errors during webhook exports (webhook listeners)

The final category includes any error that occurred while processing webhook adaptor data. The execution is marked as completed, but with errors on it, which can be retried later.

Note: You can't retry errors for jobs that fail or are canceled. If a job completes successfully, but with errors, you can retry the errors from integrator.io after fixing the cause of the error. This might include fixing the data being processed, the configuration of the flow, the export, the import, or the connections. You might also need to address an issue with a third-party endpoint.

Review errors

You can see the list of jobs that were executed for a given flow the status summary in the integration dashboard. Expand the row for a specific job to review all child jobs associated with each export, lookup, or import stage of the flow.

The Errors column lists the error count for each job during the execution of the flow. Hover over the number of errors for a View link that opens the errors associated with the job.

The following columns are displayed:

• Resolved? – Lists records that are marked as resolved
• Source – The application where the error occurred
• Code – The error code
• Message – A detailed error message that helps you understand the cause of the error
• Time – The date and time when the error occurred
• Actions – The link to view and edit retry data associated with the error

Click the edit   button under the Actions column to view the retry data for the given error. If the retry data needs to be fixed before retrying the failed errors, you can modify the record and save your modifications here.

If the number of errors associated with a job is above 1,000, then integrator.io will not allow you to view the errors in the dashboard. Instead, you can download all the errors as a CSV file, to help you further process the errors.

Error notification

You can configure notification settings to receive an email when an execution of a flow has errors from the Notifications tab for the integration. This email notifies you of any failures happening in your data flow so you can promptly take needed actions.

Manage and handle errors

You can mark errors as resolved or retry errors in an integration dashboard. You can do this for a set of selected errors or for all errors.

When you retry errors, integrator.io retries the lookup or the import execution based on point of failure. For example, if a response mapping error occurred, the snapshot of the data at the time the error was detected is stored as retry data. When you retry the job, integrator.io starts from where the flow failed and continues the flow from that point forward. If the original job included successfully completed data for the import or lookup in its original execution, the successfully completed records are not included in the retry attempt.

Once a retry attempt is successfully completed, integrator.io sends the data to the next flow step in the sequence if the flow step was configured to pause records on errors.

integrator.io does not retry errors marked as resolved, but will mark those errors as fixed, reduce the error count, and increment the resolved count of the job. If you don't need to fix a set of errors or the errors were already fixed outside of integrator.io, mark those errors as resolved. Then, the integration dashboard is free of unwanted errors, so that any future errors will be clearly visible.

Note: All jobs and their associated errors are automatically deleted after 30 days from the date they were generated.

Resolve or retry selected errors

If the total error count of a job is less than or equal to 1,000, those errors can be resolved or retried.

From the red Errors link, you can review errors individually and mark them as resolved after error correction.

1. Click the checkbox next to the error you want to mark as resolved. You can choose multiple errors at once.
2. Click Mark resolved at the top of the error dialog box. The button also displays the number of errors you've selected.
3. Run the flow again.

To resolve or retry selected errors when the error count is greater than 1,000, you must download the error CSV file first. Click Download all errors and mark the errors as retried or resolved, then upload the error file with the Upload processed errors button.

The following sample error file was downloaded from the integration dashboard. The first two errors have been marked as resolved while the next two have been marked for retry. All other errors will remain unchanged. Notice that both resolve and retry column titles should remain unchanged.

After you upload the error file, a preview of the uploaded error file displays and prompts you for a confirmation.

Resolve or retry errors in bulk

To resolve or retry all errors associated with a selected job, click the Actions overflow menu (...) and select Retry or Mark resolved.

To resolve or retry all errors found across all child jobs of a flow execution, click Retry all, Mark resolved, Run flow, Download diagnostics, or Edit flow.

To resolve or retry all associated jobs (or a set of selected jobs), click the checkbox next to the flow(s) and perform the operation.

For more information on handling errors in integrator.io, visit the Community Forum. You can ask questions and post answers.