File transfers

Contents

• File providers
• File formats
• File sizes
  • Maximum file size
  • Maximum batch size
  • Maximum record size
• File encoding
• File encryption and decryption
• File compression

File providers

During a flow, you can transfer any file from these provider endpoints:

• Amazon S3
• Google Drive
• Salesforce – as a blob (binary data embedded within export or lookup) only
• NetSuite – as a blob only
• Microsoft Azure
• FTP – including SFTP/FTPS

During a Data Loader import, you can transfer any file from these provider endpoints:

• Google Drive
• Salesforce – as a blob only
• NetSuite – as a blob only
• Microsoft Azure

File formats

When exporting or downloading files from supported providers, you can read the file and parse it. You can also create files in these formats.

• EDI X12
• EDIFACT
• Fixed-width
• JSON
• XLSX
• XML
• SOAP
• CSV (or any delimited text file)

For more information:

• Complete list of supported EDI and fixed-width templates

Known limitation: Binary (blob) data is transferred as is and is never parsed into records to add to the flow.

File sizes

Maximum file size

In general, there is no practical limit to the size of the file you can export, with certain exceptions.

Maximum batch size

• 1,000 files – downloaded or uploaded per batch

Maximum record size

• 5 MB – per flow step, such as an export or import

The Celigo platform will return an error for any record processed in excess of the 5 MB limit, whether the record was exported from an API call or a record within that limit had additional data appended, such as in a lookup step or a hook, that caused it to exceed 5 MB. In such cases, try to reduce the columns queried, further limit the record, or perform other operations in another flow or step.

File encoding

The following formats are available by default or where selectable, such as in FTP file transfers:

• UTF-8
• UTF-16LE
• Windows-1252

File encryption and decryption

You can use PGP encryption – in ASCII Armor or binary format – to sign, encrypt, and decrypt files, directories, and other types of data communication, to and from the following file providers:

• Microsoft Azure
• FTP
• Google Drive
• Amazon S3

File compression

Supported format:

• Gzip

Known limitation: ZIP and similar compression formats