Before you begin
Set up a connection to MySQL.
Tip: Click 
to open the Build SQL query editor and ask Celigo AI to generate a SQL query for you. Enter a description of your SQL query in plain English in the space provided, and Celigo AI will attempt to generate a SQL query based on your prompt.
Use bulk insert SQL query
The bulk insert data option is ideal for large data volumes. integrator.io builds the insert query for you automatically for each batch of records. Each batch's default number of records bound is 100, but you can use the Batch size field to tune your imports as needed.
By using metadata, you can now:
- Search for a specific database, dataset, table, or column to add to your query.
- Use lookups to find data.
- Refresh your data to get new updates automatically.
- Use Mapper 2.0 to map imported records.
- The metadata search will fail if you have a database or table that contains a period ( . ).
- You can use any schema and table name regardless of what you used in your connection with the following format in your import’s Destination table search: database.tableName.
Use SQL query once per record
Execute a SQL query once for each record.
You can write your SQL command in the SQL query text field. Click Edit (
) to the right of the text field to open the SQL Query builder AFE.
Use SQL query once per page of records
Execute a SQL query once per page of data. You can write your SQL command in the SQL query text field. Click Edit () to the right of the text field to open the SQL Query builder AFE.
Use SQL query on first page only
Use this option if you want to run a SQL command once per flow run. The SQL will execute as soon as the first page of data reaches the flow step. This option should be used carefully to avoid race conditions in your flows.
Troubleshooting
Question: I am getting an invalid_parameter error while exporting data from NetSuite to PostgreSQL.
Answer: When you are exporting data from NetSuite to a database, such as PostgreSQL, ensure that you enable Skip grouping in NetSuite export. By default, NetSuite results are returned as an array of arrays, whereas integrator.io expects a simple (not nested) array of results. When you enable Skip grouping, the results are received as a simple array.
Comments
Hi Kathyana Rule
After the recent release. We are getting this error and all the imports have been errored out due to the same. It says to use one of the four available import options and we are doing the same. But still it pops up with this.
Hi Yash Kumar,
Let me reach out to some people internally and get back to you!
Thanks!
Hi Kathyana Rule
I appreciate your prompt response. Good news— the issue resolved itself, and all the flows are back to their usual functioning. It seems like I encountered a sporadic issue following the recent release, but it's reassuring to know that it got fixed on its own may be something happed in the Backed.
Thanks again for your assistance!
Hi Yash Kumar,
This issue seems a bit complicated, so I've created a ticket for you (Ticket No. 201005). I'll follow the ticket so we can update the article as soon as possible. If you could comment on the ticket and provide the import ID, it would help a lot. You can find the import ID as follows:
The URL will look like this
In this case, your import ID will be 5c•••••••••••••••397c8
–– OR ––
The URL will look like this:
In this case, your import ID is 60df••••••••••••3a7e
Let me know if you need anything else. For any documentation-related questions, you can also reach out to writers@celigo.com. Thanks!
Hi Yash Kumar,
Just saw your response! I'm glad the issue resolved itself. You can either close the ticket by commenting on it, or you can leave it open so our internal team can review the issue and check that there isn't something bigger happening in the background. It's completely up to you.
Thanks!
Hi
I kindly want to know if I am correct here:
1. Bulk SQL Method: This method is optimized for large data volumes. It automatically creates a bulk insert SQL query for each batch of records(batch is per page of records), which is ideal for efficiently inserting large numbers of records into a database in fewer transactions. It only uses INSERT.
2. SQL Query Once per Page of Records: This option allows you to execute a custom SQL query for each page of data. It provides more flexibility in terms of the SQL operations you can perform (not limited to just bulk inserts) and is useful when you need to run specific SQL commands on each batch of data.
Hi Nuri Ensing
1. Bulk SQL Method: You are correct in that the Bulk SQL method creates a bulk insert SQL query for each batch of records. I believe each batch is per page of records, but I think Rohit Prasad might be better suited to answer that.
2. SQL Query Once per Page of Records: This executes a custom SQL query once per page of data. Slightly different than a custom SQL query for each page of data. From what I understand, it allows you to run one query for each page, as opposed to a different query for each page. Is that correct, Rohit Prasad?
This is my understanding, but Rohit may have more in-depth knowledge, so I would wait for his confirmation. Thanks!
Nuri Ensing you are correct in your understanding.
Hi Team,
Can I know which type supports both create and update operations in the Import MYSQL ?
Thanks in advance.
Grandhi Mounica any of the options where you write your own query support update/insert by using your system's merge operation. In MySQL's case, it would be something like this: https://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html
Hi Tyler Lamparter ,
Thanks for the response. I wrote the SQL query using the merge operation, but it is not working.
I am getting the below error. Can you please check and provide the solution for this
We received an error code "ER_PARSE_ERROR" from the MySQL indicating the submitted query "MERGE INTO xxnemko_timeentry_projecttasklist_internal_timesheet AS target USING (SELECT '1189284' AS InternalID, '45390.667' AS ProjectActualWork, '' AS ProjectExternalID, 'PRJ0014213' AS ProjectID, '4- In Progress' AS ProjectStatusText, 'Nemko Foundation : Nemko Group AS : Asia : Nemko Korea Co., Ltd' AS ProjectSubsidiary, '15' AS SubsidiaryID, '1189284' AS Proj...
Thanks,
Mounica G
Grandhi Mounica you'd need to use SQL syntax that MySQL supports:
Please sign in to leave a comment.