Optimize multiple connections in your integration app flows for better throughput

A connection is a fundamental resource that regulates the flow of data to and from the source and destination. When you query an app to export data from it or import data into it, that HTTP request is added to the connection’s queue and processed in order, after earlier requests.

Prior to integrator.io v6.11 (released in November 2020), integration apps were allowed only one connection per endpoint account (with the exception of the Amazon – NetSuite integration app with two NetSuite connections). Now, by adding additional connections to a single endpoint, you can exercise greater control over the volume of data that is requested through the connections. This flexibility is key to getting high-volume flows to run more quickly.

When you set up multiple connections for flows based on logical prioritization rules and available “concurrency,” you can optimize the connections for improving data throughput to your API app, file server, or database system.

Concurrency acts as a queue, determining the maximum number of requests that a connection can run in parallel. All requests within a single queue are sent in parallel, and therefore one connection has the potential to act as a gateway – and on occasion, as a bottleneck – for processing large sets of data from multiple queries. In addition, an app’s API policy can impose restrictions on concurrency, also known as governance or rate limits.

Contents

• Create a connection
• Replace all connections throughout a flow
• Replace the connection in a flow step
• Best practices for utilizing multiple connections

Create a connection

To add a new connection to an existing integration app:

1. Sign in to your integrator.io account and navigate to your integration app tile.
2. Open the Connections tab.
3. Click + Create connection.

4. On the Create connection: application page, select the additional endpoint connection you wish to configure. For example, in the Shopify – NetSuite integration app, you will be able to add new connections only for Shopify and NetSuite endpoints.
5. On the Create connection page, configure the connection with the required settings. For more assistance, refer to Connector solutions.

Tip: For each connection, provide a unique name and description that will make it easy to identify later. 

After you configure the connection successfully, a new connection is added to the Connections tab.

Replace all connections throughout a flow

You can replace the connection only for the flow steps in the flow that uses a particular connection.

Tip: Be sure to configure the new connection before you continue with the steps below.

1. Sign in to your integrator.io account and navigate to your integration app tile.
2. Navigate to Flows.
3. In the Actions column of a flow, click the overflow (...) menu Edit flow to proceed to Flow Builder.

4. Open the Connections tab in Flow Builder.
5. In the Actions column of the connection, you want to modify, click the overflow (...) menu, and select Replace connection from the drop-down list.

6. In the Replace connection pane, the Connection drop-down list contains all available connections for this endpoint. Choose a different connection from the list.

7. Click Replace.

All jobs in progress will be canceled, including any jobs that happen to be currently queued to run, even if the actual processing has not yet started.

Replace the connection in a flow step

1. Sign in to your integrator.io account and navigate to your integration app tile.
2. Go to Flows, and select the flow that you want to modify.
3. In Flow Builder, click any step, such as an import, export, or listener.
4. From the Connections drop-down list, select a new connection. (If you successfully added a new connection, it is automatically reflected in the Connections list if it applies to the current flow.)

Best practices for utilizing multiple connections

• Do not replace a connection within an integration flow that is in progress. If you replace the connection, the integration app automatically cancels all the jobs for the affected flow steps.
• Celigo does not recommend replacing the connection for the following integration flows: 
  • Shopify
    • Shopify Order to NetSuite Order Add 
    • Shopify Order to NetSuite Cash Sale Add
    • Shopify Order to NetSuite Order/CashSale Add
  • Amazon
    • Amazon (MFN) Order to NetSuite Order Add
    • Amazon (FBA) Order to NetSuite Order Add
    • Amazon Settlement Report to NetSuite Custom Settlement Records
  • Square
    • Square Order to NetSuite Cash Sale 
• If any of the integration flow belongs to a chain of flows (means two or more flows are auto-triggered one after another), it is recommended to maintain the same connection across all chained flows.
  • Order + customer sync in any eCommerce integration app.

Example: The customer sync is a low concurrency flow and order sync is a high concurrency flow. In this case, you should use the same connection for these two flows. If you use a different connection, the order sync fails and is blocked.

• • Estimate + customer sync in the Salesforce – NetSuite integration app.
• In the Amazon – NetSuite integration app, it is recommended to run all the settlement integration flows with the same NetSuite connection.
• In the Amazon – NetSuite integration app, if you wish to assign a secondary connection for a flow, it is recommended to select the appropriate flow from the Select flows to allot secondary NetSuite connection setting (Settings > General section > General subtab)
• If you clone the Salesforce – NetSuite integration app after adding multiple connections, all the connections are cloned from the source tile to the destination tile.
• If you have multiple stores/accounts/banks within an integration tile, any newly added connection is applicable for all the stores/accounts/banks.
• For the following integration flows, if you add an additional lookup, import, or export flow step, it is recommended to use the same connection for all flow components (imports or exports). 
  • Shopify
    • Shopify Order to NetSuite Order Add 
    • Shopify Order to NetSuite Cash Sale Add
    • Shopify Order to NetSuite Order/CashSale Add
    • Shopify Payout Transactions to NetSuite Deposit
  • Amazon 
    • NetSuite Transfer Order to Amazon Inbound Shipment Plan