This article describes how to export files from Azure Data Lake Storage Gen2 (ADLS Gen2) and look up files and records on a per-record basis.
Create an Azure Data Lake Storage Gen2 export
Start creating an export from Azure Data Lake Storage Gen2 in either of the following ways:
- From the Build menu, select Flows > Create flow. Then, click Add source.
– or –
- From the Resources menu, select Exports. Then, click + Create export.
From the Application drop-down list, select Azure Data Lake Storage Gen2 and your available Azure Data Lake Storage Gen2 connection.
Note: Execution logs aren't currently available for Transfer files out of source application or Look up additional files (per record) steps. Look up additional records (per record) steps do support execution logs, since they run through integrator.io's HTTP-based framework.
Edit Azure Data Lake Storage Gen2 export details
General
Name (required): Provide a clear and distinguishable name. You will have the option to choose this export throughout integrator.io, and a unique identifier will prove helpful later when selecting among a list of exports that you've created.
Description (optional): Describe your export so that you and others can quickly understand its purpose. Be sure to highlight any nuances that a user should be aware of before using this export in a flow. As you make changes to the resource, be sure to keep this description up to date.
Connection (required): The Azure Data Lake Storage Gen2 connection to use for your export.
Parse files being transferred: Set to Yes if you want integrator.io to parse the file contents into structured records. If set to No, files are transferred as raw binary data without parsing.
How would you like to parse files?
This section appears only if Parse files being transferred is set to Yes.
File type (required): Choose the type of file to be exported from Azure Data Lake Storage Gen2. The file type you select changes the fields available in the export configuration. Acceptable file types include:
- CSV (or any delimited text file)
- EDI X12
- EDIFACT
- Fixed Width
- JSON
- XLSX
- XML
CSV (or any delimited text file)
Sample file (that would be parsed) (required): Choose a sample file to define the record structure. Click Choose file and navigate to a sample version of the files you will be exporting.
CSV parser helper: Use the CSV parser helper to visualize and experiment with how integrator.io parses CSV files into JSON records. See CSV parser helper.
EDI X12
EDI X12 format (required): Select the EDI X12 file format that matches the files you are exporting from Azure Data Lake Storage Gen2.
EDIFACT
EDIFACT format (required): Select the EDIFACT file format that matches the files you are exporting from Azure Data Lake Storage Gen2.
Fixed Width
Format (required): Select the Fixed width file format that matches the files you are exporting from Azure Data Lake Storage Gen2.
JSON
Sample file (that would be parsed) (required): Choose a sample file to define the record structure. Click Choose file and navigate to a sample version of the files you will be exporting.
Resource path: Optionally define the JSON path to the resources you are exporting from the JSON file using handlebars syntax.
XLSX
Sample file (that would be parsed) (required): Choose a sample file to define the record structure. Click Choose file and navigate to a sample version of the files you will be exporting.
File has header: Check this box if the files you are exporting contain a top-level header row.
XML
Sample file (that would be parsed) (required): Choose a sample file to define the record structure. Click Choose file and navigate to a sample version of the files you will be exporting.
XML parser helper: Use the XML parser helper to visualize and experiment with how integrator.io parses XML files. See XML parser helper.
Where would you like to transfer files from?
Container name (required): Enter the name of the Azure Data Lake Storage Gen2 container to export files from. This is the same container you see listed under your storage account in the Azure Portal. (The underlying Azure DFS REST API calls these "filesystems" — the terms are interchangeable, but integrator.io uses "container" to match the Azure Portal.) You can also select a container from the drop-down list, which is populated by calling Azure's List Filesystems API against your connection. If the list comes back empty — including when your Service Principal has access scoped to specific containers rather than the whole storage account — the drop-down simply shows no results; this isn't an error, so type the container name directly instead. You can use handlebars syntax to set the container name dynamically.
Directory path: Enter the path within the container to transfer only the files stored at that location. For example, if the full path is my-container/reports/monthly/, enter reports/monthly/ as the directory path. If left blank, all files in the container are eligible for transfer. Click Browse to navigate the container's folder structure, or click the handlebars button to build a dynamic path. Because ADLS Gen2 supports real directories, an empty folder still appears in the Browse picker — unlike flat-namespace file stores where empty "folders" don't actually exist. If the handlebars-resolved directory doesn't exist at runtime, the export still finishes successfully with zero records rather than failing — useful for scheduled flows targeting a date-partitioned path like inbound/{{date 'YYYY/MM/DD'}}/ before that day's folder has been created.
Include subdirectories: Check this box to include files in every subdirectory beneath the directory path, not just the immediate level. When checked, integrator.io calls Azure's List Path API with recursive=true, which walks the entire subdirectory tree server-side in a single call. When unchecked, only files directly inside the directory path are included.
File filtering conditions: See Create file filtering conditions for FTP exports.
How would you like to group and sort records?
Sorting and grouping allows you to manage your flow's files by sorting and grouping records by field. You can group records by field (column1, column2, column3…) and sort them in ascending or descending order.
Advanced
Decrypt files: This option is enabled if your connection is configured to use PGP encryption/decryption. When enabled, integrator.io decrypts files after downloading them from ADLS Gen2.
Decompress files: Set to True if you are exporting compressed files. integrator.io decompresses the files after downloading them.
Leave file on server: When set to False, integrator.io deletes each file from the container after it is successfully transferred. Set to True to leave files in place after transfer.
Backup container: Enter the name of the ADLS Gen2 container where integrator.io should archive a copy of the exported file. Leave blank to skip backup. If specified without a Backup path, the original file path is preserved in the backup container. Supports handlebars syntax.
- If you specify a Backup container without a Backup path, the source path is preserved in the backup container. For example, a file at input/filename.json is archived to backupContainer/input/filename.json.
- If you specify both a Backup container and a Backup path, the file is written to exactly that target, with no source-path prefix applied. For example, backup container backupContainer with backup path archive/filename.json writes to backupContainer/archive/filename.json.
Backup path: Enter the full path for the backup copy, including any folder path. Supports handlebars syntax for dynamic naming. You must specify a Backup container to use this field — a path without a container is invalid and will prevent the export from saving.
File encoding: Select the character encoding for exported files. Defaults to UTF-8. Other supported values include Windows-1252 and UTF-16LE.
Page size: Enter the number of records to include in each page of export results. Defaults to 20.
Data URI template: Enter a template to construct a data URI for each exported file. Supports handlebars syntax.
Override trace key template: Enter a template to override the default trace key for exported records. Supports handlebars syntax.
Note: If a listed file is in Archive tier, the export reports a clear error asking you to rehydrate the file in Azure before integrator.io can read it, rather than surfacing Azure's raw error response.
Note: If the handlebars-resolved path contains \, :, or ?, or exceeds 1,024 characters, the export fails with a message naming the specific problem.
Custom settings
Click Launch form builder to create a custom settings form for this export, or edit the Settings JSON directly.
Simple / HTTP view
Every record-based lookup step includes a Simple / HTTP view toggle. Simple view shows the curated Resource and API endpoint selectors described above. HTTP view exposes a raw request builder (method, relative URL, query parameters, headers, body) that inherits your connection's authentication and token refresh, so you can call any Azure DFS API endpoint — including POSIX ACL operations, lease management, and blob-tier transitions — without waiting for these to be added as curated options.
Save your export
- Click Save to save your current settings.
- Click Save & close to save your changes and exit the page.
- Click Close to exit without saving.