Articles in this section

Consolidate errors in a single report

Anitha Abraham
Updated

Retrying errors for flows can become tedious, especially when you have many integrations and each integration has at least 2 or more flows. If you have to access each flow manually and retry the error data, the process becomes tedious and time-consuming. This issue can be resolved by installing and running the Consolidated errors report flow.

This is a custom flow and needs to be run manually. This flow consolidates errors that have an Unresolved status for all flows in the selected domain for the selected time period (within the last 30 days or more, depending on your data retention plan). The consolidated errors will be added to a report that includes the retry status and the API used to retry the errors. The consolidated errors report will be saved to an FTP server.

Limitations:

  • The start date for consolidating and retrying the error data must be within the last 30 days or earlier depending on your data retention plan.

  • The error report (in CSV format) generated needs to be stored in a FTP location. You may need to create a FTP endpoint, if one doesn’t already exist.

  • We recommend that you run this flow with a time duration of 2 to 3 days each time you run the flow. For example, if you want to consolidate the errors for the last 7 days, we suggest that you break the time duration into 2 halves and run the flow two times for each half. You would have 2 consolidated error reports for the two time periods.

  • You need to select the time duration and the domain to consolidate the errors each time you run the flow.

Install and configure the flow

Before you install the Consolidated errors report flow, download the zip file to your local drive.

Download

You need to install and configure the custom flow in order to run it. To install and configure the custom flow, follow the steps mentioned in the Install integration section and Configure integration section of the Install integration or download integration zip article.

Run the flow manually

  1. Access the Consolidated errors report tile.
  2. Access the Settings tab to customize the time duration and the domain to consolidate the errors.
    Consolidated errors report Settings.png

    • Start date and time: Select the start date and time to review consolidated errors. Select the start time in the UTC time zone. The start date can be within the last 30 days or earlier depending on your data retention plan. For example, if you want to review errors that have occurred since 08:00 AM EST 01/31/2023; you can select the start date and time as 01:00 PM UTC 01/01/2023.

    • End date and time: Select the end date and time to review consolidated errors. Select the end time in the UTC time zone. The end date can’t exceed the present date and time. For example, if you want to review errors that occurred between the Start date and time and 08:00 AM EST 01/31/2023; you can select the end date and time as 01:00 PM UTC 01/31/2023.

      You cannot select the end date and time as 01:01 PM UTC 01/31/2023 or 01:00 PM UTC 02/01/2023.

      You can select a date which is before the end date and time. For example, 01:00 PM UTC 01/30/2023 or 00:59 AM UTC 01/31/2023.

    • Domain to consolidate errors: Enter the domain to consolidate errors – integrator.io or eu.integrator.io.

  3. If you are running the flow for the first time, you need to perform the actions described below. If not, you can skip this step and proceed to Step 4.

    1. Access the Flows tab and enable the flow.

    2. Click the flow name to view the various steps involved in the flow.

      Edit FTP details.png

    3. In the Destinations & Lookups section, select and drag the mouse to the left till you view the Generate error report step.

    4. Click Transfer to access the Edit transfer pane and specify the FTP location where the report needs to be saved.

      Select FTP location.png

    5. Enter the FTP path in the Directory path setting.

    6. Click Save to update the FTP location where the consolidated errors report needs to be saved.

    7. Go back to the Flows tab.

  4. Run the flow.

  5. An error report will be generated in CSV format with all the details including a link to retry the error.

    Note

    The consolidated errors report will be saved to the FTP endpoint specified in the Directory Path setting of the Generate error reports step in the Consolidated errors report flow.

  6. Scroll to the retryDataLink column in the report of the error you want and click on the link. You will be redirected to the error management screen of the flow to which the error relates to.

  7. If you click a link in the retryDataLinkAPI column, you can view the details of the error received in JSON format.

Ignore errors in sandbox environment

When the Consolidated errors report flow is run, it also consolidates any errors for the flows which are in the sandbox environment. To ensure that errors related to the sandbox environment are not added to the consolidated errors report, perform the actions described below.

To add the output filter, you need to modify the input data in the following manner:

  1. Access the Flows tab and click the flow name to view the various steps involved in the flow.

  2. In the Get all flows step, click the Define options button (Define options).

    The Output filter button (output_filter.svg) will appear after the Get all flows step.

    Output filter added.png

  3. Click the Output filter button to access the Define output filter pane.

    Modifying_input_data.png

    The parameter “sandbox”: “” (line 4 in the image above) needs to be added in the “record” array to be able to add the record.sandbox option in the Rules section.

To add the output filter to ignore the sandbox records,

  1. In the Rules section of the Define output filter pane, select the record.sandbox option from the options available in the drop-down list.

    1. Click the Settings button to the right of this field.

    2. Select the Operand type as Field and Data type as String in the Operand Settings dialog box.

    3. Click Save to confirm the changes.

  2. Select the not equals option in the drop-down list that appears after the record type is selected.

  3. Enter the value as true in the text box that appears after the record type is selected.

    1. Click the Settings button to the right of this field.

    2. Select the Operand type as Value and Data type as String in the Operand Settings dialog box.

    3. Click Save to confirm the changes.

      The output filter will be displayed as shown in the following image:

      Output filter for sandbox records.png

    4. Click Save & close to confirm the changes made and return to the flow.

  4. Run the Consolidated errors report flow.
    The consolidated error report will be generated without any sandbox records added to it.

Schedule the flow to run automatically

You have the option to schedule the Consolidated errors report flow to run every day at a specified time. The frequency to consolidate the errors automatically can be customized using presets from 15 minutes to once a day or by selecting a duration of your choice. To know how to schedule the order data flow as per your business needs, see Scheduling flows.

Tip

We recommend that you set the flow to be automatically run once every 24 hours to avoid duplication of errors.

Limitation: When you choose to schedule the flow to run automatically, errors can be consolidated only for a period of up to 24 hours before the current date. If you schedule the flow to be run on January 02, 2023 at 08:00 AM EST, the flow will fetch errors from 08:00 AM EST January 01, 2023 to January 02, 2023.

Before you can schedule the flow to run automatically, you need to modify the search parameters in the Get errors step of the Consolidated errors report flow using the actions described below:

  1. Access the Flows tab and click the flow name to view the various steps involved in the flow.

  2. In the Get errors step, click Lookup to access the Edit lookup pane.

  3. Click Launch in the What would you like to export? section to edit the Search parameters.

    Update Search parameters to schedule flow.png

  4. Replace {{dateAdd settings.integration.gte}} with {{dateAdd (dateFormat) -86400000}} in the occurredAt_gte setting.

  5. Replace {{dateAdd settings.integration.lte}} with {{dateAdd (dateFormat)}} in the occuredAt_lte setting.

  6. Click Save to update the search parameters.

  7. Click Save & close to update the changes made in the lookup.

    You can now schedule the flow to be run once every 24 hours and any errors found in this time duration will be added to the error report and saved in the selected FTP location.

Run the flow manually

If you have scheduled the flow to run automatically and want to revert to running the flow manually, you can do so by updating the search parameters in the Get errors step of the Consolidated errors report flow.

  1. Access the Flows tab and click the flow name to view the various steps involved in the flow.

  2. In the Get errors step, click Lookup to access the Edit lookup pane.

  3. Click Launch in the What would you like to export? section to edit the Search parameters.
    Update Search parameters to run flow manually.png
  4. Replace {{dateAdd (dateFormat) -86400000}} with {{dateAdd settings.integration.gte}} in the occurredAt_gte setting.
  5. Replace {{dateAdd (dateFormat)}} with {{dateAdd settings.integration.lte}} in the occuredAt_lte setting.

  6. Click Save to update the search parameters.

  7. Click Save & close to update the changes made in the lookup.

    You can now run the flow manually. When the flow is run, the error report will be generated and saved in the selected FTP location.

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.