Articles in this section

Consolidate integration errors in a single report

Balasubramanian Mahadevan
Updated

When you manage many integrations, each with several flows, retrying errors flow by flow is slow. The Consolidated errors report flow collects every error with an Unresolved status from all flows in a domain, for a time period you choose, into one CSV report saved to an FTP server. Each row includes the retry status and links you can use to retry the error.

Limitations

  • The start date must be within the last 30 days, or earlier depending on your data retention plan.
  • The report is saved as a CSV file to an FTP location. If you don't have an FTP endpoint, create one first.
  • Run the flow for 2 to 3 days at a time. For example, to consolidate errors for the last 7 days, split the period in two and run the flow once for each half. You get one report per period.
  • You select the time period and domain each time you run the flow manually.

Install and configure the flow

  1. Download the Consolidated errors report flow zip file to your computer.
  2. Install and configure it by following the Install integration and Configure integration sections of Install integration or download integration zip.

Run the flow manually

  1. Open the Consolidated errors report tile.
  2. On the Settings tab, set the time period and domain:
    • Start date and time: The start of the period, in UTC. It can be within the last 30 days, or earlier depending on your data retention plan.
    • End date and time: The end of the period, in UTC. It can't be later than the current date and time.
    • Domain to consolidate errors: Enter integrator.io or eu.integrator.io.
  3. The first time you run the flow, set the FTP location for the report. Otherwise, skip to the next step.
    1. On the Flows tab, turn on the flow, and then select the flow name.
    2. In Destinations & Lookups, scroll to the Generate error report step and select Transfer.
    3. In Edit transfer, enter the FTP path in Directory path, and then select Save.
    4. Return to the Flows tab.
  4. Run the flow.

The flow saves a CSV error report to the Directory path set in the Generate error report step. In the report:

  • Select a link in the retryDataLink column to open the error management screen of the flow the error belongs to.
  • Select a link in the retryDataLinkAPI column to view the error details in JSON.

Exclude sandbox errors

By default, the flow also collects errors from flows in the sandbox environment. To leave them out, add an output filter to the Get all flows step.

  1. On the Flows tab, select the flow name.
  2. On the Get all flows step, select Define options, and then select Output filter.
  3. In Define output filter, add the parameter "sandbox": "" to the record array in the input data. This makes record.sandbox available in the rules.
  4. In Rules, select record.sandbox. Select Settings next to it, set Operand type to Field and Data type to String, and then select Save.
  5. Select the not equals operator.
  6. Enter the value true. Select Settings next to it, set Operand type to Value and Data type to String, and then select Save.
  7. Select Save & close.

The next time you run the flow, the report contains no sandbox errors.

Schedule the flow to run automatically

You can schedule the flow using presets from every 15 minutes to once a day, or a custom frequency. See Scheduling flows. When scheduled, the flow collects errors from the 24 hours before each run. For example, a run on January 2 at 8:00 AM EST collects errors from 8:00 AM EST on January 1. Schedule the flow to run once every 24 hours to avoid duplicate errors.

Before you schedule the flow, change the search parameters in the Get errors step:

  1. On the Flows tab, select the flow name.
  2. On the Get errors step, select Lookup.
  3. In Edit lookup, under What would you like to export?, select Launch to edit the Search parameters.
  4. In occurredAt_gte, replace {{dateAdd settings.integration.gte}} with {{dateAdd (dateFormat) -86400000}}.
  5. In occuredAt_lte, replace {{dateAdd settings.integration.lte}} with {{dateAdd (dateFormat)}}.
  6. Select Save, and then select Save & close.

Switch back to manual runs

To run the flow manually again after scheduling it, restore the original search parameters in the Get errors step:

  1. On the Flows tab, select the flow name.
  2. On the Get errors step, select Lookup.
  3. In Edit lookup, under What would you like to export?, select Launch.
  4. In occurredAt_gte, replace {{dateAdd (dateFormat) -86400000}} with {{dateAdd settings.integration.gte}}.
  5. In occuredAt_lte, replace {{dateAdd (dateFormat)}} with {{dateAdd settings.integration.lte}}.
  6. Select Save, and then select Save & close.