This article describes how to import files to an FTP server.
-
From the Build menu, click Flow builder > Add destination/lookup.
– or –
From the Resources menu, click Imports > + Create import.
-
From the Application list, select FTP.
-
Create or select an FTP connection.
-
If asked, click Transfer files into destination application and create a transfer or use an existing transfer.
Create transfer panel
Name your transfer (required): Provide a clear and distinguishable name. You will have the option to choose this import throughout integrator.io, and a unique identifier will prove helpful later when selecting among a list of imports that you’ve created.
Description (optional): Describe your resource so that you and others can quickly understand its purpose. Be sure to highlight any nuances that you or a another user should be aware of before using this import in a flow. As you make changes to the resource, be sure to keep this description up to date.
Connection (required): The FTP connection to use for your import. You can create your connections in the Resources menu → Connections or click the + icon to create a new connection. Each connection contains credentials and other related information needed to access your FTP server.
One to many (required): Choose Yes (advanced) if you have a single record that will create multiple records. You would use this setting if, for example, your parent record contains child records you need to process. This setting cannot be used when importing a CSV file.
Generate files from record (transfer only): Choose Yes to build a new CSV, XML, JSON, or another file. Choose No to transfer raw files as-is (PDF, JPG, and so forth).
File type (required): Select the type of file that you want to generate. For example, choose CSV if you are importing a flat, delimited text file, or XLSX for a binary Microsoft Excel file. The file name you specify can include a file extension, but it does not affect this setting. You can then choose a sample file to define the record structure. For example, if you are importing CSV files (i.e. files containing Comma Separated Value data), choose CSV. Each file type includes additional settings to customize your import. Acceptable file types include:
-
CSV (or any delimited text file)
-
EDI X12
-
EDIFACT
-
Fixed width
-
JSON
-
XLSX
-
XML
Based on the file type, the settings you see will differ. See the settings in a file type from the grouped sections below.
CSV (or any delimited text file), XLSX
The CSV and XLSX file types have similar settings for this section. The CSV file type has more settings that you can configure.
Sample file (that would be generated): If you have a sample file that represents the record structure of the files to transfer, you can upload the sample file here. We will use the sample file to auto-set various fields (where possible), and also help you map data in a subsequent step. The sample file that you provide does not need to be overly large (at max 5 records), but it should contain all the fields that you want to work with, and also be in the same format that the transfer will need to generate when running in a production capacity.
Include header: Check the box to
CSV: set the very first row in your CSV file as column names so that anyone reading the CSV file can quickly understand what each column represents). It's selected by default.
XLSX: (optional) set the header in your XML file that specifies the XML version number, and optionally the character encodings, as the XML declaration on the first line of the document.
How many files would you like to generate?: (optional) Select how the flow should generate output files. Select One file per flow run to create a single file that contains all of the records processed during the flow run. If the flow includes more than one export, one file will be generated for each export. For real-time flows, a file is created using all of the records received in the current request. Select One file per page of records to generate a separate file for each page of records processed during the flow run.
CSV generator helper: (optional – CSV only) The CSV generator helper can be used to visualize and experiment with how integrator.io builds CSV files (or any other delimited text files) from the JSON records/rows being processed by your flow.
Column delimiter: (optional – CSV only) Select the delimiter that separates fields (or values) in a record. While comma is the default separator, it could also be any of the other options provided in the list.
Row delimiter: (optional – CSV only) Select the delimiter that separates the end of a row (or record). While the default separator is line feed or new line, it could also be any of the other options provided in the list.
Replace new line with space: (optional for CSV only) Check this box only if you want to replace new lines in the content of the data (except rowDelimiters) with a space.
Replace tab with space: (optional – CSV only) Check this box oly if you want to replace tabs in the content of the data (except columnDelimiters) with a space.
Truncate last row delimiter (optional – CSV only) Check this box only if you want to prevent errors that may occur if a string value in the CSV file exceeds the maximum allowed length.
Wrap with quotes: (optional – CSV only) Check this box only if you want to wrap all the values in the file with quotes.
Custom header rows:(optional – CSV only) In some rare cases, it is necessary to include one or more custom header rows before the actual CSV data. Use this field to add any such prefix rows.
See also File generation for structured data in CSV and XLSX.
JSON, XML
The E and XML file types have the same settings for this section.
Sample file (that would be generated): If you have a sample file that represents the record structure of the files to transfer, you can upload the sample file here. We will use the sample file to auto-set various fields (where possible), and also help you map data in a subsequent step. The sample file that you provide does not need to be overly large (at max 5 records), but it should contain all the fields that you want to work with, and also be in the same format that the transfer will need to generate when running in a production capacity.
How many files would you like to generate?: (optional) Select how the flow should generate output files. Select One file per flow run to create a single file that contains all of the records processed during the flow run. If the flow includes more than one export, one file will be generated for each export. For real-time flows, a file is created using all of the records received in the current request. Select One file per page of records to generate a separate file for each page of records processed during the flow run.
EDI X12 & EDIFACT
The EDI X12 and EDIFACT file types have the same settings for this section.
EDI profile: Select an existing EDI profile, or click the plus sign (+) to create a profile. An EDI profile in Celigo is a centralized configuration that maps and enforces the specific rules, guidelines, and document structures for a specific trading partner (such as a retailer, distributor, or 3PL). It acts as a single source of truth to translate and validate business documents (like 850s, 856s, and 810s) between your ERP and your trading partner.
Generic definition: Select a document type to convert the canonical data to an EDI document to use in your flow’s import.
EDI generator helper: Launch the EDI generator helper to use file definition rules and check the EDI files generated. The EDI file generator helper allows you to modify the templated rules to solve more specific file generation requirements. Within the generator helper, the rules panel specifies and generates EDI files from the JSON records/rows being processed by your flow. You can use the sample flow data and generated file panels to test your rules.
How many files would you like to generate?: (optional) Select how the flow should generate output files. Select One file per page of records to generate a separate file for each page of records processed during the flow run. Select One file per key to generate a separate file for each document type defined in the ST segment (Transaction Set Header) during the flow run.
Fixed Width
Fixed width file type has the same settings for this section.
Format: Select the document type for a trading partner.
File generator helper: Launch the file generator helper to use file definition rules and check the files generated. The file generator helper allows you to modify the templated rules to solve more specific file generation requirements. Within the generator helper, the rules panel specifies and generates EDI files from the JSON records/rows being processed by your flow. You can use the sample flow data and generated file panels to test your rules.
Directory path (required): Specify the directory path FTP folder where you want to transfer your files. For example, if you set this to 'MySite/Items' integrator.io will first look for a parent folder 'MySite', and then for a child folder 'Items', and then transfer all files into this child folder 'Items'.
File name (required): Specify how the files uploaded to the FTP site should be named.
Tip
You can type '{{{' to use handlebars to include a predefined timestamp or random unique file identifier (UUID) in your file name.
Timestamp handlebar example: If you type FileXYZ-{{{timestamp "YY-MM-DD" "America/Los_Angeles"}}}.txt , the files will be named with the pattern: FileXYZ-16-06-30.txt.
Verify that your timestamp format is filesystem compatible. Incompatible ASCII characters include:
-
< (less than)
-
> (greater than)
-
: (colon)
-
" (double quote)
-
/ (forward slash)
-
\ (backslash)
-
| (vertical bar or pipe)
-
? (question mark)
-
* (asterisk)
Random handlebar example: If you type FileXYZ-{{{random "UUID"}}}.txt , the file will be named with the following pattern: FileXYZ-69368e91d9a440f79165b73afd46859d.txt generating a unique ID (UUID) of the file. The length of the UUID defaults to 32 if left unspecified, so typing FileXYZ-{{{random "UUID" 16}}}.txt would limit the UUID to 16 characters.
You can also include whatever file extension you like, and the file extension will never change the type of data generated.
You can add mock response data to imports in the Mock response section of the Edit import page as simulated response data when configuring a flow.
Encrypt files (optional): Use this cryptographic system to encrypt generated files. This option is enabled only when you have configured at least one cryptographic system in the connection. If it is unchecked, then encryption will not be performed on generated files.
Use temp file while upload in progress: Check this box if the destination folder where your file is being generated is also being watched by another service. If checked, integrator.io will "hide" the file being generated by integrator.io until it completes.
In progress file name: Enter a temporary file name. The Celigo platform writes the file to the name you specify while the write operation is in progress. Upon completion, the Celigo platform renames this file to the intended filename defined in the File name setting.
Number of files per batch (optional): Enter the maximum number of files (from 1 to 1000) that can be processed in a single batch request. This setting doesn’t limit the total number of files that you can process in a flow; instead, it allows you to optimize for really big files where bigger batches might cause network timeout errors. If you are importing very small files, increasing the batch size to include several files at a time will improve flow performance.
Backup files path (optional): Specify the directory path of the FTP folder where files will be backed up after a successful transfer.
File encoding (optional): The file encoding indicates how the individual characters in your data are represented on the file system. Leave this blank if you do not know what the encoding format is. The default encoding is UTF-8 and the other supported format is: UTF-8, Windows-1252 and UTF-16LE
Compress files (optional): Compresses your imported data as a .zip file.
Invoke (uneditable): For use with the integrator.io API, this resource can be invoked via an HTTP request to this unique URL.