Q: When importing large volumes of data into Snowflake using a batch insert SQL query, how do I optimize the performance if the batch insert is slow?
A: We recommend that you use the optimized bulk load rather than the batch insert option in your Snowflake import. This option aggregates data before loading, reducing the need for numerous batch inserts and significantly lowering costs by minimizing warehouse compute time. Therefore, it's also the easier and quicker option.
If for some reason, your data flow to Snowflake is running slow, consider configuring the following flow settings to optimize the performance:
-
Set a higher page size on your export
-
Increase the target concurrency level of your flow connections.
Comments
Please sign in to leave a comment.