Articles in this section

Transformation 2.0 overview

Transformation 2.0 (Rules 2.0 in the Define transformation editor) is the rules-based transformation option you use to build output records field by field on exports and lookups. You can process nested arrays at any depth, combine values from several input fields, compute values with handlebars expressions, translate values with lookups, and convert records into rows. This article explains the concepts you need before you build a Rules 2.0 transformation: JSON structure, field transformation types, how to select input values, input context, and data type conversion.

Note: Transformation 2.0 is not available for an import step.

Also see the Celigo University course Exploring Flow Builder.

JSON structure

The JSON structure setting determines how the output record relates to the input record. Select it from the toolbar at the top of the Rules 2.0 editor.

JSON structure What the output contains Use it when
Modify input record { } The full input record, plus the changes your rules make. Fields without a rule pass through unchanged. You need a few targeted changes, such as adding one field or reformatting another, and want to keep everything else.
record { } Only the fields you define. Fields without a rule are dropped. You want a new record built from scratch, or you want several sources to produce the same standard (canonical) format.
rows [ ] An array of rows built from each input record. You need row-based output, such as building a CSV file in which rows that share a key column stay together in the same file.

Field transformation types

Each output field has a Field transformation type that defines where its value comes from. Set it in the field's settings.

  • Standard — Copies the value of an input field to the output field. Standard is the default type.
  • Hard-coded — Writes the same fixed value to the output field for every record. For object and object array fields, you can enter a static JSON object or array of objects.
  • Lookup — Replaces input values with output values from a table you define (static lookup) or from a lookup cache. For example, every time the input value is apple, the output value is banana.
  • Handlebars expression — Builds the output value with a handlebars template. Use it to concatenate fields, perform calculations, encode or decode strings, or read values from custom settings.

The types available for a field depend on its output data type. For example, object fields don't offer Lookup. For the full list and every related setting, see Transformation 2.0 field settings.

Select input values with JSONPath or handlebars

Rules 2.0 selects input values with JSONPath and builds custom values with handlebars expressions.

JSONPath identifies one or more fields in the input record, such as $.customer.email or $.items[*].sku. JSONPath supports simple paths and advanced queries, including array slices and recursive descent. You can enter several comma-separated paths in the Input column of one row to build an array from different parts of the record, for example $.owner_name,$.owner_id,$.owner_phone.

Handlebars expressions produce a value that isn't a single input field. Use them to join fields, multiply numbers, format several fields together, or read flow and integration settings. For example, {{record.firstName}} {{record.lastName}} combines two fields into one string.

Input context

The Input context setting determines the root of the data your rules read.

  • record — The root is the record itself. You reference fields directly, for example $.field. Use this context when your rules only need the record's business data.
  • envelope — The root is an object that wraps the record with runtime data: job details, settings, query parameters, request or response headers, export timestamps, and file metadata. You reference record fields as $.record.field and runtime data as $.job._id, $.settings.flow.myValue, or $.fileMeta.fileName.

For every object in the envelope and the step types that provide it, see Transformation 2.0 input context reference.

Data types and automatic conversion

Every output field in Rules 2.0 has a data type: String, Number, Boolean, Date, Object, String array, Number array, Boolean array, or Object array. Input fields use the same types except Date. Rules 2.0 supports recursive definitions, such as an object array whose objects contain object arrays.

When the input and output data types differ, Rules 2.0 converts the value where it can. For example, a number mapped to a string output field becomes a string. Some combinations can't be converted, and you can't save the field until the types are compatible. For every valid and invalid combination, see Data type validation.

Rules 2.0 and Mapper 2.0

Rules 2.0 transformations and Mapper 2.0 mappings use the same engine, field types, and field settings. A transformation shapes data as it leaves an export or lookup. Mapper 2.0 shapes data into the format a destination application accepts on a lookup or import. For destination-specific settings, see Mapper 2.0: Destination field settings.