Articles in this section

Transform data in integrator.io

A transformation restructures the records in a flow so that later steps receive data in the shape they need. You can flatten nested JSON, rename fields, drop fields you don't need, build arrays, or convert several source formats into one standard format. This article explains where transformations run in a flow, when to use one instead of a mapping, and how to choose between Rules 1.0, Rules 2.0, and JavaScript. It's for anyone who builds or maintains flows in integrator.io.

How transformations fit into a flow

A transformation is an optional setting on a flow step. It runs on the data that leaves that step, before the data reaches the next step.

Flow step Where the transformation runs Transformation options
Export On records after the export retrieves them Rules 1.0, Rules 2.0, JavaScript
Lookup On records the lookup returns Rules 1.0, Rules 2.0, JavaScript
Import On the import result data, after the import submits records Rules 1.0, JavaScript

On every step, you add a transformation from the step's Outgoing data settings. For the steps, see Add a transformation to a flow step.

Transformations and mappings

A transformation and a mapping both change data, but they do different jobs. A transformation changes the structure of a record: it flattens, nests, renames, and removes fields. A mapping, which you configure on a lookup or import, assigns source fields to the fields a destination application expects.

You don't always need a transformation. For a flow with one source and one destination, a mapping on the import is usually enough. A transformation is the better choice in these situations:

  • One source, many destinations — An export returns large records with nested objects and arrays within arrays. A transformation reduces each record to the fields that every later step needs, so you don't repeat that work in each import's mapping.
  • Many sources, one destination — Several exports return records in different formats. A transformation on each export converts its records into one standard (canonical) format, so the destination step processes every record the same way. Adding a new source later means writing one more transformation, not changing the destination.

Choose a transformation option

integrator.io offers three transformation options, selected at the top of the Define transformation editor. Rules 1.0 is also called Transformation 1.0, and Rules 2.0 is also called Transformation 2.0. A pre-save page hook is a fourth way to change exported data, configured separately from the transformation.

Option How it works Best for Consider
Rules 1.0 You list rules that copy input fields to output field paths. Only the fields you list appear in the output. Renaming, flattening, and removing fields with a short list of rules. Import-result transformations. Every field you want to keep needs a rule, even unchanged fields.
Rules 2.0 You build an output structure field by field, sourcing values with JSONPath, handlebars expressions, hard-coded values, or lookups. Nested arrays at any depth, computed values, data type conversion, and record-to-row output. Rules 2.0 has more settings to learn than Rules 1.0.
JavaScript You write a transform function that receives each record and returns the transformed record. Conditional logic and restructuring that rules can't express. Requires JavaScript knowledge.
Pre-save page hook A script that runs on each page of records after an export or lookup retrieves them. Work across records: merging, deleting, adding, or sorting records within a page. Requires JavaScript knowledge and works on a page of records, not a single record.

Rules 1.0 and Rules 2.0 are both supported. Choose based on the structure you need to produce.

A flow step applies only the option selected when you save. For what happens to the other options, see Add a transformation to a flow step.

Rules 2.0 and Mapper 2.0

Rules 2.0 transformations and Mapper 2.0 mappings share the same engine and field settings. For how they differ, see Transformation 2.0 overview.