Articles in this section

Map data with Mapper 2.0

Mapper 2.0 shows the destination JSON structure you're building next to a sample of your source data, so you can map fields, nest objects and arrays, and preview the result in one panel. This article is for builders who configure imports to HTTP and REST API-based applications or file transfer destinations. It covers the panel, the toolbar, and the actions that apply to every mapping.

Open Mapper 2.0

  1. Open the flow in the flow builder.
  2. On the import step, select More elipses-button.png > Incoming data > Mapping mapping-icon.png.
  3. To switch between versions, select Mapper 1.0 or Mapper 2.0 at the top of the panel.

A banner reminds you that mappings you enter in Mapper 2.0 apply when you save, even if you switch back to Mapper 1.0. To learn which mapper opens and which one runs, see Mapping types and versions.

Panel layout

Area What it shows
Toolbar Controls for navigating and filtering the mapping. See Toolbar.
Field mappings The destination fields you're building, their data types, and the source field or value for each.
Input A sample of the source data, headed Source record { }.
Output The destination data you're building, headed Destination record { }. Select Preview to refresh it.

Toolbar

Control What it does
Expand all and collapse all Show every field with all subfields, or only top-level fields.
JSON structure Sets the destination structure to record { } or rows [ ]. See Choose the JSON structure.
Input context Sets the source data to record or envelope. See Choose the input context.
Refresh fields Reloads the source fields so recent changes to your sample data appear.
All fields Filters the rows. See Filter and search fields.
Search Finds source or destination fields by name. Press Cmd+F (Mac) or Ctrl+F (Windows) to select it. Adding a row closes search.
More actions Auto-populates destination fields, removes all mappings, or removes all non-mandatory fields. Removing all mappings also removes inactive mappings.

Turn on Auto preview in the list beside Preview to refresh Output after every change.

Map a field

Each row maps one destination field to a source field. In the row:

  1. Enter or select the destination field name. Start typing to find fields from the destination application.
  2. Select the data type of the destination field. See Data type validation reference.
  3. Select the source field. The source is shown in JSONPath notation, for example $.user.location. See JSONPath notation.
  4. To change how the value is produced or what happens when it's empty, select Settings settings-icon.png. The indicator beside Settings shows the mapping type: S for standard, E for handlebars expression, DL for dynamic lookup, SL for static lookup, and an icon for lookup cache. Enter hard-coded values in double quotes. Mapper 2.0 reads any other value that isn't a JSONPath as a handlebars expression. See Mapper 2.0 field settings.

You can save a destination field without a source field. You can't save a source field without a destination field.

Row controls

Control What it does
Drag handle Reorders rows within the same level. You can't drag a row to another level.
Add row Adds a row at the current level of the destination structure.
Delete Deletes the row. It appears when you point to the row.
Handlebars editor Opens the editor for a handlebars expression. In the editor, you can use Create or explain handlebars using Celigo Ora to build the expression.
Actions Clones the row, including objects and object arrays, or marks the mapping inactive.

An inactive mapping stays in the panel but doesn't appear in the preview or run. Making a parent row inactive also makes its child mappings inactive. Mapper 2.0 deletes empty rows when you save.

Choose the JSON structure

JSON structure sets the shape of the data that integrator.io sends to the destination:

  • record { } builds one JSON object.
  • rows [ ] builds an array of objects.

With rows [ ], the mapper tree shows rows [ ] as a locked root entry with a blank Destination value. This entry is the array itself, not a row you map to, so you can leave it blank.

With record { }, the top entry is an ordinary unnamed row, and it needs a Destination value. If it's blank, Save & close fails with "One or more destination field values not entered." The message doesn't say which row is missing a value, so check every row, including the top one. If you rebuild a mapping under the other structure, check the structure first when you see this error.

integrator.io recognizes the structure of your source data automatically. For definitions and samples, see Work with JSON and file output. For the effect on output, see Mapper 2.0 examples.

If a mapping produces an empty output, such as {}, [], or [null], integrator.io ignores the record when it sends data to file providers, databases, warehouses (bulk and batch inserts), and HTTP imports that don't have a post map hook. The record isn't inserted or passed to later steps. If the import has a post map hook, your script decides how to handle these records.

Choose the input context

Input context controls what the source data includes and how you reference fields.

Context Source data Reference a field
record The record from the source application only. $.field
envelope The record plus run information: $.job.*, $.settings.*, $.queryParams.*, $.pathParams.*, and, on file provider export and lookup steps, $.fileMeta.*. $.record.field, $.job.*, and so on

Use envelope when you need run information, such as settings or job details, without writing handlebars expressions.

File metadata on file provider steps

When a file provider export or lookup step has a transformation and Input context is envelope, the data includes a fileMeta object. It's available for FTP, SFTP, FTPS, Amazon S3, AS2, Google Drive, Box, Dropbox, NetSuite File Cabinet, and HTTP file steps.

Field Type Description
fileName String The name of the source file.
lastModifiedTime String (ISO 8601) When the file was last modified.
filesize Number The file size in bytes.

For example, reference the file name with {{fileMeta.fileName}}, or with $.fileMeta.fileName when Input context is envelope. When one run processes several files, each record gets the fileMeta of the file it came from. Existing preSavePage scripts that access file metadata continue to work unchanged.

Filter and search fields

To find required or unmapped fields in a large structure, set All fields to Required fields or Mapped fields, and then apply the filter. A padlock marks each required field. In a filtered view, you can't change mappings or add fields. Switch back to All fields to edit.

Inactive mappings appear in search results but not in a filtered view. Empty destination fields also appear in search.

Auto-populate and auto-map destination fields

For HTTP and REST-based applications, and for file transfer imports with a sample file, integrator.io can build the destination structure and map it for you.

  1. Select More actions > Auto-populate destination fields to generate the destination structure with the application's expected field names.
  2. Select the destination fields to map. Selecting a parent field selects all the fields below it.
  3. Select Auto-map selected destination fields.

Auto-map fills only fields that don't have a source field. It doesn't overwrite mappings you entered. The option is available only for new mappings and fields, and while it's active, actions that change the destination structure are turned off. For AI help with mapping, see Celigo Ora overview.

To start over, select More actions > Remove all mappings. To keep only required fields, select Remove all non-mandatory fields.