This feature allows you to identify existing records when importing records. It analyzes data files and datasets to identify similar or existing records in the destination application and react accordingly. You can use this for imports from relational databases and HTTP APIs including:
Records are normally imported through create, insert, or update methods. The process of importing records varies from application to application, but you can typically insert or update a record in a database and create or update records in HTTP or REST API-based applications.
There are two ways to import your records, depending on the type of connection you create. For relational databases that use an on-premise agent, you'll be able to insert, update, or insert or update the data.
Note
Oracle DB (SQL) is the exception to this rule, as it does not use insert, update, or insert or update import query types.
For example, you can select Insert to add new records to the database or Update to revise existing records in the same import. After you import records, integrator.io checks each record (based on your specified parameters) and determines whether it is new or existing. See Step B for information about how to identify an existing record.
Import record in a relational database using an on-premise agent
For relational databases that use a cloud-based connection, you'll be able to use Use bulk insert SQL query (recommended), Use SQL query once per record, Use SQL query once per page of records, or Use SQL query on first page only to insert or update your records. Note that your options may vary based on the database you choose. To learn more about importing records using cloud-based connections, navigate to the specific connection's import documentation.
Import records in a cloud-based relational database (Microsoft SQL pictured)
As an example, you can select the Composite HTTP method, which gives you access to three relevant sub-methods:
Composite method and types
When you choose this method, integrator.io searches for and compares records based on the settings you specify in the Identify existing records section. When it finds new, non-existent data, it creates new records. And, when it finds existing records in the destination application, it ignores the records.
When you choose this method, integrator.io searches for and compares records based on the settings you specify in the Identify existing records section. When it finds new, non-existent data, it creates new records. And, when it finds existing records in the destination application, it updates the records with the new data.
When you choose this method, integrator.io searches for and compares records based on the settings you specify in the Identify existing records section. When it finds new, non-existent data, it ignores the records. When it finds existing records in the destination application, it updates the records with the new data.
After you select an import method, you can choose how you would like to identify the existing records. Regardless of the application or import method, you have two options to identify the records.
-
Records have a specific field populated
– or –
-
Run a dynamic lookup
Options for identifying existing records
This method searches existing records using a field you specify. It determines if the field in your source record already has a value; if so, the source record is considered an existing record.
Relational database with a specific field populated
HTTP or REST API with a specific field populated
This method searches the destination application to find a record quickly. If a record matches the search criteria, the source record will be considered an existing record. These dynamic searches are called ‘lookups’.
Learn more about creating lookups.
Dynamic lookup in a relational database
Dynamic lookup in an HTTP or REST API
Comments
Are you planning to implement this for Snowflake imports?
Please sign in to leave a comment.