Introduction

In the world of data integration, file size limits can often be a concern. However, with integrator.io, there are generally no practical limits to the size of the file you can process. This article will provide you with detailed information about the few exceptions to this rule.

File size limits

While integrator.io generally supports processing extremely large files (i.e. 200 MB files, or even larger for some use cases), here are the few exceptions worth noting.

1. Parsing individual EDI / fixed width files: The maximum limit is 10 MB.
2. Parsing individual XLSX files: The maximum limit is 100 MB.
3. Parsing individual XML files: The maximum limit is 5 MB.
4. Generating individual EDI / fixed width files: The maximum limit is 5 MB.
5. Generating individual XLSX files: The maximum limit is 100 MB.
6. Generating individual XML files: The maximum limit is 5 MB.

Understanding integrator.io file processing

integrator.io is capable of downloading or uploading files from a wide range of applications and protocols, including FTP, Amazon S3, Google Drive, and many more.

When downloading files, integrator.io offers two options:

1. The files can be parsed into records for further processing.
2. The files can be left as-is in the form of blobs.

Similarly, when uploading files, integrator.io provides two alternatives:

1. The files can be generated from records in your flow.
2. The files can be uploaded as-is from blobs that were first downloaded by your flow.

It's important to note that uploading and downloading files is different from working with HTTP-based APIs where records are also being parsed and generated. The content in this article is specifically relevant to downloading, uploading, and processing standalone files, and not to working with records and HTTP-based APIs.

Conclusion

While integrator.io offers extensive flexibility in file processing, it's essential to be aware of the file size limits for different file types. Understanding these limits will help you plan your data integration processes more effectively.