Leverage multiple connections in your Integration App flows to improve throughput

Every Integration App starts with at least one default connection to each endpoint, which you configure during installation. It’s important to keep that original connection online and available so that your flows and any background processes are able to reach the source and destination apps. 

The connection is the primary integrator.io resource through which all requests and incoming data are regulated. Optimizing the flow of data through multiple connections is your most powerful strategy for speeding up high-traffic flow steps and taking advantage of maximum concurrency (the limit of simultaneous requests that an API imposes) for the most important data.

Contents

• Troubleshoot slow flows
  • 1. Identify the bottleneck
  • 2. Verify the existing configuration
  • 3. Try adding one new connection and adjusting the concurrency levels for each connection to that endpoint
  • 4. Borrow concurrency from the “main” connection
• Examples for optimizing connections
  • Example A: Single Integration App, giving priority to order-related flows
  • Example B: Multiple Integration Apps borrowing concurrency from a connection in the Integration App with the highest priority data
• Appendix: Governance policies for common apps

Troubleshoot slow flows

Before adding connections, walk through the following checklist to try to narrow down the causes of any flows running below peak performance. 

The information-gathering steps (1-4) and the solutions provided proceed from simplest to the most complex. 

1. Identify the bottleneck

• Is it happening with NetSuite, another endpoint, or both?
  • Try to isolate where the highest volume of data and most processing occurs.
  • Review the performance explanations and tips at Fine-tune integrator.io for optimal data throughput.
  • Are you retrieving additional fields, searches, files (or blobs) – or whole catalogs – unnecessarily?
    

• Are the timestamps on your flow run dashboards indicating longer than expected flow steps?
  • Typically, exports run much faster than imports. A single request to retrieve a database table can quickly return 10 or 1,000 records. However, importing 1,000 updated records requires at least 1,000 separate requests, and sometimes many more.
  • Lookup steps are designed to send requests one at a time, not in parallel, and could run slowly, depending on the payload size.
    

• Does the flow seem to be slowing down over time?
  • Once you’ve identified a likely culprit, compare that flow or flow step with historical jobs for the same Integration App.

2. Verify the existing configuration

• Are the connections’ concurrency levels set to a lower value than what the external system can support?
  • If so, increase the levels and run the flows before continuing.

• Are the Export types set correctly?
  • Selecting Delta, as opposed to All, for example, has performance implications. 

• Is a flow consistently slow, or might there be some temporary or cyclical factor at fault?
  • Monitor the integration’s performance for a day or two and then decide the next steps.

3. Try adding one new connection and adjusting the concurrency levels for each connection to that endpoint

• Identify the priority of the respective flows within your business rules.
  • The slowest flows might not be the ones most in need of attention. Start by ordering your Integration App in relative importance for your company’s integration needs. The following list represents a typical hierarchy, from highest to lowest priority:
    1. Orders
    2. Inventory
    3. Fulfillments
    4. Billings 
    5. Product 
    6. Settlement

• Create a second connection to that endpoint.
  • This secondary connection is dedicated to forming smaller queues of requests, as the result of sharing a lesser portion of the total concurrency available for the endpoint.
  • Replace the original connection for lower priority flows with this new connection.
  • Start small to isolate the variables, and then test the effects. 

• Allocate higher concurrency levels within the connection that serves high-priority flows.
  • See the example, below, Single Integration App, giving priority to order-related flows.

• Repeat this step, as needed, with a third connection – or more. 
  • Keep in mind that adding too many connections to the same account with low concurrency levels might have adverse effects on performance.

4. Borrow concurrency from the “main” connection

Note: This step is recommended when you have more than one Integration App installed in your integrator.io account or you have built custom flows within the App. Borrowing concurrency is an advanced option, requiring deliberate application and repeated testing.

• If you are running multiple Integration Apps, repeat troubleshooting Step 3 to add a new connection to each Integration App.

• Choose one Integration App’s connection as the “main” connection.
  • Then, point all other Integration Apps’ connections for that endpoint to borrow concurrency from the main connection.
  • See the example, below, Multiple Integration Apps borrowing concurrency from a connection in the Integration App with the highest priority data.

Examples for optimizing connections

The general guidelines, demonstrated below, can be summarized as follows:

1. Create a secondary connection – called, for example: “NetSuite (Low Traffic).”
2. Parcel out the total available NetSuite concurrency:
   1. Dedicate a generous share to your default NetSuite connection.
   2. Give the remaining concurrency to the new secondary connection.
3. Replace the connection on all low-priority flows (all except Order flows) with NetSuite (low traffic).
4. Borrow concurrency, if you have more than one Integration App.

Refer back to the data you gathered during the troubleshooting discovery to see how a solution might be tailored for your company’s apps, data, and flows. 

• Example A: Single Integration App, giving priority to order-related flows
• Example B: Multiple Integration Apps borrowing concurrency from a connection in the Integration App with the highest priority data

Example A: Single Integration App, giving priority to order-related flows

First, let’s examine a Shopify – NetSuite Integration App, with the default flows and connections:

The connections are already set to the maximum concurrency allowed in this customer’s accounts. 

Suppose that the holiday season is expected to bring 50,000 new orders and 100,000 item updates per day. With 20,000 items listed in Shopify, the increased traffic will invariably slow down the inventory, pricing, and item flows.

One solution is to add connections, as follows, favoring traffic in the order-related flows and especially the heavy payloads in the Shopify imports:

Example B: Multiple Integration Apps borrowing concurrency from a connection in the Integration App with the highest priority data

In the next scenario, the integrator.io customer has a total of four Integration Apps running daily, all querying NetSuite frequently:

• Shopify – NetSuite
• Amazon Seller Central – NetSuite
• eBay – NetSuite
•  3PL Central – NetSuite

The NetSuite account allows up to 15 concurrent requests, which are apportioned among the four connections, as follows:

The inefficiency in this configuration may not be apparent at first glance, but it has the effect of limiting the maximum concurrency for any one of the Integration Apps. Let’s assume that Shopify is most active, and the other flows don’t run as often. At no point can the Shopify flows access the full available concurrency; they can have only 5 concurrent requests at a time. The concurrency in the other connections is effectively wasted when not in use. 

The simplest solution is to give all the concurrency to one connection and distribute the rest by letting the connections in the remaining Integration Apps borrow from the “main” connection:

* A connection that borrows concurrency from another cannot have its own concurrency level. That setting is disabled. 

Then, to give priority to certain flows and distribute concurrency where needed, the customer can add secondary connections for low-priority or low-volume flows or steps. Then, apportion the different levels from the two “pools” by borrowing from the two “main” connections.

Appendix: Governance policies for common apps

Adding new connections can provide higher concurrency – and therefore better throughout when distributed among your flows – only if the endpoint permits it. As seen in the following table, these policies vary according to the vendor, as well as the API account type that you subscribe to. For example,

• Adding more NetSuite, Amazon, or Salesforce connections will not increase the total concurrent requests available to you, because these apps enforce global concurrency limits per account
• Some apps like Shopify allow you to configure custom apps that can increase concurrency per connection, thereby improving throughput
• Other apps limit you to a daily maximum and will return errors only when that limit is exceeded