Sort and group records

Contents

• Sort by
• Group by
• Sorting and grouping matrix
• Sorting and grouping with new and existing flows
  • Sort and group in a new flow
  • Edit sort and group in a new flow
  • Edit sort and group using the deprecated feature

The Sorting and Grouping option allows you to manage your flow’s files by sorting and grouping records by field. This feature allows you to aggregate your data based on your chosen field(s), and sort the records in ascending or descending order (ASC or DESC).

Sort and group (only file provider exports)

Group (HTTP, database, and data warehouse exports and lookups)

Sort by

This feature allows you to sort by multiple fields (ASC or DESC) or in the order that the fields were initially entered. You can enter the “Sort records by fields” by a field ID or by using a dot notation.

Group by

Rate this feature
Included in the Nov. 2022 release You can group records when parsing files from any file source, or when exporting records from HTTP, databases, or data warehouses. This feature allows you to aggregate your data based on your chosen field(s). Select the fields you want to group, based on your sample response data. Fields other than basic data types (such as string and number) are unavailable. If the source application does not sort the exported data, then grouping may not work as expected. The order of each grouping is always respected. You can enter records by field id, by using a dot notation, or one field at a time. When you input at least one field, the platform will group records in the order that the fields were initially entered, from top to bottom.

Note: The dot notation starts from “record”: { in the example below.

Take the following JSON record:

{
  "page_of_records": [
    {
      "record": {
        "NAME": "Sarah",
        "AGE": 97,
        "PURCHASE": "Beach towel",
        "CATEGORY": "Summer"
      }
    },
    {
      "record": {
        "NAME": "John",
        "AGE": 98,
        "PURCHASE": "Ski gloves",
        "CATEGORY": "Winter"
      }
    },
    {
      "record": {
        "NAME": "Ana",
        "AGE": 99,
        "PURCHASE": "Beach ball",
        "CATEGORY": "Summer"
      }
    },
    {
      "record": {
        "NAME": "James",
        "AGE": 100,
        "PURCHASE": "Snowboard",
        "CATEGORY": "Winter"      
       }
    }
  ]
}

Using the example JSON record, you can group by the Category field. Your data is grouped into Summer and Winter rows. Of course, integrator.io can handle more detailed groupings, say if you want to group records by Category and Purchase, or by Category, Age, and Purchase. If you’re working with a file provider export, you can sort the records by Ascending or Descending.

{
  "page_of_records": [
    {
      "rows": [
        {
          "CATEGORY": "Summer",
          "NAME": "Sarah",
          "AGE": 97,
          "PURCHASE": "Beach towel"
        },
        {
          "CATEGORY": "Summer",
          "NAME": "Ana",
          "AGE": 98,
          "PURCHASE": "Beach ball"
        }
      ]
    },
    {
      "rows": [
        {
          "CATEGORY": "Winter",
          "NAME": "John",
          "AGE": 99,
          "PURCHASE": "Ski gloves"
        },
        {
          "CATEGORY": "Winter",
          "NAME": "James",
          "AGE": 100,
          "PURCHASE": "Snowboard"
        }
      ]
    }
  ]
}

In some cases, you might have CSV file exports or lookups:

In this case, you’d group by column (Category), and each row would be grouped into the Summer or Winter category. Logically, it would look something like this:

Tip: Remember that the Page size property under the Advanced section determines how many records/groups can go into a page. If a single group exceeds the page size, that group is discarded from the export, and an error indicating this issue is reported.

Sorting and grouping matrix

This matrix details all possible sorting and grouping outcomes.

Sorting and grouping with new and existing flows

This section provides details regarding sorting and grouping new and existing flows. 

Sort and group in a new flow

When sorting and grouping files in a new lookup or export:

1. Select a Sorting and grouping option to specify the file transfer requirements.

Note: Deprecated grouping fields are not available when creating a flow.

2. Specify whether the file has headers.
3. Sort and group records as needed.
4. Select Save and close.

Edit sort and group in a new flow

When editing sorting and grouping in a new flow:

1. Select a Sorting and grouping option to specify the file transfer requirements.

Note: Deprecated grouping fields are not available when creating a flow.

2. Specify whether the file has headers.
3. Sort and group records as desired.
4. Select Save and close.

Edit sort and group using the deprecated feature

When editing the sorting and grouping in an existing flow:

• The deprecated fields are available until you start using the new feature.
• Existing, deprecated group fields are automatically generated in the new feature, though they may require further configuration.
• After the new fields are transferred and saved, the deprecated fields will disappear permanently. The only way to use the old fields is through the integrator.io API.