Import data into PostgreSQL

Before you begin

Set up a connection to PostgreSQL.

Tip: Click to open the Build SQL query editor and ask Celigo AI to generate a SQL query for you. Enter a description of your SQL query in plain English in the space provided, and Celigo AI will attempt to generate a SQL query based on your prompt.

Use bulk insert SQL query

The bulk insert data option is ideal for large data volumes. integrator.io builds the insert query for you automatically for each batch of records. Each batch's default number of records bound is 100, but you can use the Batch size field to tune your imports as needed. There is a query limit of 100KB, so for larger batch sizes, the number of records imported in a single request is adjusted to the size limit.

By using metadata, you can now:

• Search for a specific database, dataset, table, or column to add to your query.
• Use lookups to find data. 
• Refresh your data to get new updates automatically.
• You can also use Mapper 2.0 to map imported records.

Note: You can use any schema and table name regardless of what you used in your connection with the following format in your import’s Destination table search: database.schema.table name.

Caution: Celigo platform integrator.io can’t process databases with a dot in the name. To work around this, create a connection with the database (name with a dot in it) and the required schema (database.schema.). Then, in the Destination table import field, enter the table instead of database.schema.table.

Use SQL query once per record

Execute a SQL query once for each record.

You can write your SQL command in the SQL query text field. Click Edit () to the right of the text field to open the SQL Query builder AFE.

Use SQL query once per page of records

Execute a SQL query once per page of data. You can write your SQL command in the SQL query text field. Click Edit () to the right of the text field to open the SQL Query builder AFE.

Use SQL query on first page only

Use this option if you want to run a SQL command once per flow run. The SQL will execute as soon as the first page of data reaches the flow step. This option should be used carefully to avoid race conditions in your flows.