Generate JSON or XML files with handlebars for transfers

When a file-based import writes JSON or XML files, then define each file's structure with a handlebars template. This article is for builders who send records to FTP, SFTP, Amazon S3, Azure Blob Storage, or Google Cloud Storage and need control over the file's structure - for example, a single JSON array for an API, JSON Lines for a data lake, or a well-formed XML document for a trading partner. integrator.io doesn't add array brackets or a root element to your output. To produce a JSON array, include [ and ] in your template.

This article explains how integrator.io combines pages of records into files, which template pattern produces each file structure, and what each template warning or error means.

Prerequisites

• A flow with an export and a file-based import.
• A working connection to the file-based destination.
• Permission to edit the flow.

Consider existing JSON or XML file imports

Consider your existing JSON or XML imports with handlebars templates in the light of the following points:

• For JSON and XML templates that generate one file per page of records, an {{#each}} loop and a footer are now optional.
• Templates that already wrap a JSON array or XML root around an {{#each}} block will continue to work. You do not need to change them.
• If you had added a placeholder footer such as {{myfakefooter}} so the flow would run, now you can delete it. However, leaving it does not break the file.
• You still must type [ and ] if you want a JSON array. integrator.io does not add them.

Configure new JSON or XML file imports

You set up JSON or XML file generation on the import step of your flow.

1. On the Home page, select your integration, and then your flow. 
2. In the flow builder, select the file-based import step.

3. In the Edit transfer form, under How would you like to generate files? set File type to JSON or XML.

   

4. In How many files would you like to generate?, select One file per flow run or One file per page of records.
5. In JSON document (JSON) or the XML template field, enter the handlebars template.
6. Select Preview to check the output.
7. Select Save.

Preview uses one sample page of records and doesn't upload a file. A test run uploads to the real destination, so use Preview, not a test run, to check if you're getting the required output for your destination application.

Combine pages and generate files for imports

integrator.io sends records to an import in pages. A page is a batch of records, and the export's page size (Advanced → Pages) sets how many records each page holds. And, the import's How many files would you like to generate? decides what happens to those pages - if you select 

• One file per flow run - integrator.io combines all pages into one file.
• One file per page of records - integrator.io writes each page as a separate file and doesn't combine them.

For One file per flow run, integrator.io splits your JSON or XML template into three parts before it runs:

• JSON templates can use {{#each batch_of_records}} to loop over records. An {{#each}} block is optional for JSON. 

• XML templates accept {{#each batch_of_records}} or {{#each data}}. XML templates that generate one file per flow run must include an {{#each}} block.

When integrator.io combines pages into one file, it joins JSON pages with a comma and concatenates XML pages with no separator.

Sample data used in article

Every example in this article uses the same five orders, split into three pages with a page size of 2:

• SO-1042 and SO-1043 on page 1, 
• SO-1044 and SO-1045 on page 2, and 
• SO-1046 on page 3.

Each item in batch_of_records has this structure. Reference a field in your template as this.record.<field>, for example this.record.orderId.

  {
  "record": {
    "orderId": "SO-1042",
    "customer": "Northwind Outfitters",
    "sku": "JKT-NAVY",
    "qty": 2,
    "total": 249.5
  }
}  

Line breaks in the output examples are added for readability. The exact whitespace in your file depends on your template.

Choose the required template pattern for your import

Choose a template pattern based on the file structure your destination expects.

Write a JSON array

To write a valid JSON array, wrap the {{#each}} loop in a header and footer that include [ and ]. This is the most common pattern, and existing templates that use it keep working.

 {
  "orders": [
    {{#each batch_of_records}}
      {{{jsonSerialize this.record}}}{{#unless @last}},{{/unless}}
    {{/each}}
  ]
}

{{{jsonSerialize this.record}}} writes each order as a JSON object. {{#unless @last}},{{/unless}} adds a comma between orders on the same page.

One file per flow run produces one file:

 {
  "orders": [
    {
      "orderId": "SO-1042",
      "customer": "Northwind Outfitters",
      "sku": "JKT-NAVY",
      "qty": 2,
      "total": 249.5
    },
    {
      "orderId": "SO-1043",
      "customer": "Blue Harbor Co",
      "sku": "HAT-OLIVE",
      "qty": 1,
      "total": 88
    },
    {
      "orderId": "SO-1044",
      "customer": "Cedar & Pine",
      "sku": "BOOT-8",
      "qty": 1,
      "total": 412.1
    },
    {
      "orderId": "SO-1045",
      "customer": "Sequoia Trading",
      "sku": "GLOVE-M",
      "qty": 3,
      "total": 54.75
    },
    {
      "orderId": "SO-1046",
      "customer": "Harborview Supply",
      "sku": "BAG-CANVAS",
      "qty": 1,
      "total": 132
    }
  ]
}

The commas after SO-1043 and SO-1045 mark the two page boundaries. integrator.io adds them automatically, so the result is one valid array regardless of how many pages the run has.

One file per page of records produces three files, each a complete, valid array:

orders-1.json

  {
  "orders": [
    {
      "orderId": "SO-1042",
      "customer": "Northwind Outfitters",
      "sku": "JKT-NAVY",
      "qty": 2,
      "total": 249.5
    },
    {
      "orderId": "SO-1043",
      "customer": "Blue Harbor Co",
      "sku": "HAT-OLIVE",
      "qty": 1,
      "total": 88
    }
  ]
}

orders-2.json

  {
  "orders": [
    {
      "orderId": "SO-1044",
      "customer": "Cedar & Pine",
      "sku": "BOOT-8",
      "qty": 1,
      "total": 412.1
    },
    {
      "orderId": "SO-1045",
      "customer": "Sequoia Trading",
      "sku": "GLOVE-M",
      "qty": 3,
      "total": 54.75
    }
  ]
}

orders-3.json

  {
  "orders": [
    {
      "orderId": "SO-1046",
      "customer": "Harborview Supply",
      "sku": "BAG-CANVAS",
      "qty": 1,
      "total": 132
    }
  ]
}

integrator.io doesn't show a warning for this template with either file-count setting. [Verify how the file name and page-number suffix are configured]

Write JSON Lines

JSON Lines writes one JSON object per line with no surrounding array. Use it when the destination, such as a data lake or Snowflake stage, expects JSON Lines. The template contains only the loop, with nothing before {{#each}} and nothing after {{/each}}:

{{#each batch_of_records}}
{{{jsonSerialize this.record}}}
{{/each}}}

integrator.io doesn't wrap the output in [ ] and doesn't treat this template differently when it joins pages. The result depends on the How many files would you like to generate? setting.

One file per page of records produces clean JSON Lines with no extra characters:

orders-1.json

{"orderId":"SO-1042","customer":"Northwind Outfitters","sku":"JKT-NAVY","qty":2,"total":249.5}
{"orderId":"SO-1043","customer":"Blue Harbor Co","sku":"HAT-OLIVE","qty":1,"total":88}

orders-2.json

{"orderId":"SO-1044","customer":"Cedar & Pine","sku":"BOOT-8","qty":1,"total":412.1}
{"orderId":"SO-1045","customer":"Sequoia Trading","sku":"GLOVE-M","qty":3,"total":54.75}

orders-3.json

  {"orderId":"SO-1046","customer":"Harborview Supply","sku":"BAG-CANVAS","qty":1,"total":132}

One file per flow run joins pages with a comma, the same as every other JSON template:

{"orderId":"SO-1042","customer":"Northwind Outfitters","sku":"JKT-NAVY","qty":2,"total":249.5}
{"orderId":"SO-1043","customer":"Blue Harbor Co","sku":"HAT-OLIVE","qty":1,"total":88}
{"orderId":"SO-1044","customer":"Cedar & Pine","sku":"BOOT-8","qty":1,"total":412.1}
{"orderId":"SO-1045","customer":"Sequoia Trading","sku":"GLOVE-M","qty":3,"total":54.75}
{"orderId":"SO-1046","customer":"Harborview Supply","sku":"BAG-CANVAS","qty":1,"total":132}

This output is neither a JSON array nor valid JSON Lines. The commas before SO-1044 and SO-1046 mark the page boundaries. This is expected behavior. If your destination can't read it, select One file per page of records, or wrap the loop in [ ] as described in "Write a JSON array".

Preview may show Evaluated result is not valid JSON for this template. The warning appears only in Preview, and you can still save and run the import.

Write a summary without a loop

A template without an {{#each}} loop writes one output per page instead of one per record. Use it for a summary, such as the number of orders on each page:

{"count": {{batch_of_records.length}} }

The template field shows a yellow warning because the template has no {{#each}} block. You can still save and run the import.

One file per page of records writes a valid file for each page of the five sample orders:

• orders-1.json: {"count": 2 }
• orders-2.json: {"count": 2 }
• orders-3.json: {"count": 1 }

One file per flow run runs the template once per page and joins the results with commas:

{"count": 2 },{"count": 2 },{"count": 1 }

This output isn't valid JSON, and integrator.io doesn't wrap it into one object. To get a single count for the whole run, add an {{#each}} loop as described in "Write a JSON array", or select One file per page of records.

Write an XML document

XML templates use the same header, loop, and footer model as JSON templates, but integrator.io concatenates XML pages with no separator. Because of this, XML templates that generate one file per flow run must include an {{#each}} block.

Template uses a loop with a root element

Wrap the loop in a root element to produce a well-formed XML document. XML templates accept {{#each data}} or {{#each batch_of_records}}:

  {{#each data}}
  
    {{this.record.orderId}}
    {{this.record.customer}}
    {{this.record.sku}}
    {{this.record.qty}}
    {{this.record.total}}
  
  {{/each}}

Double braces, such as {{this.record.customer}}, escape special characters, so & is written as &. Use double braces for XML values to keep the document valid.

One file per flow run produces one document:

  
  
    SO-1042
    Northwind Outfitters
    JKT-NAVY
    2
    249.5
  
  
    SO-1043
    Blue Harbor Co
    HAT-OLIVE
    1
    88
  
  
    SO-1044
    Cedar & Pine
    BOOT-8
    1
    412.1
  
  
    SO-1045
    Sequoia Trading
    GLOVE-M
    3
    54.75
  
  
    SO-1046
    Harborview Supply
    BAG-CANVAS
    1
    132
  

One file per page of records produces three files, each a complete <Orders> document that contains the orders from one page.

Template uses a loop without a root element

A loop with no root element writes a sequence of <Order> elements. This output is an XML fragment - a set of elements without a single enclosing root - and is valid only if the destination accepts fragments. If the destination expects a full XML document, use a root element.

Template without a loop

Whether an XML template without an {{#each}} block is allowed depends on the file-count setting:

• One file per page of records - Allowed, with the same yellow warning as JSON. Each page becomes its own valid document.
• One file per flow run — integrator.io shows an error because concatenating a complete document for each page would produce XML with multiple root elements, which parsers can't read. For example:
   <summary><count>2</count></summary>
  <summary><count>2</count></summary>
  <summary><count>1</count></summary>
   

You can still save and run. Preview and the template field show the error so you can add an {{#each}} block or select One file per page of records before the file is written. Imports created or edited through the integrator.io API don't show this UI error and can still write the unreadable file.

Warnings and errors reference

integrator.io checks JSON and XML templates when you save or preview them and shows these messages [Verify exact message text]:

Related articles

• Handlebars reference

• Import or transfer files to an FTP server