Optimize your integration app flows for throughput

Adding multiple connections to your integration app is the most effective way to increase throughput across your flows. This article helps you find what's slowing your flows, then split and share endpoint concurrency across connections so high-priority flows like orders stay fast. It ends with two worked examples you can adapt.

Troubleshoot slow flows

Before you add connections, work through this checklist to narrow down why flows are running below peak performance. The steps go from simplest to most complex.

1. Identify the bottleneck

• Is the slowdown in NetSuite, another endpoint, or both? Find where the highest volume of data and most processing occurs. Check whether you're retrieving extra fields, searches, files (blobs), or whole catalogs that you don't need. For more tips, see Fine-tune integrator.io for optimal data throughput.
• Do the timestamps on your flow run dashboards show flow steps taking longer than expected? Exports usually run much faster than imports. A single request can return 10 or 1,000 records from a database table, but importing 1,000 updated records takes at least 1,000 separate requests, and sometimes many more. Lookup steps send requests one at a time, not in parallel, so they can run slowly depending on the payload size.
• Is the flow slowing down over time? Once you've found a likely cause, compare that flow or flow step with earlier jobs for the same integration app.

2. Verify the existing configuration

• Are the connections' concurrency levels lower than what the external system supports? If so, raise them and run the flows before you continue.
• Are the export types set correctly? For example, choosing Delta instead of All affects performance.
• Is the flow consistently slow, or could a temporary or cyclical factor be the cause? Monitor the integration's performance for a day or two before you decide what to do next.

3. Add a connection and adjust concurrency levels

1. Rank your flows by business priority. The slowest flows aren't always the ones that need the most attention. A typical order, from highest to lowest priority, is: orders, inventory, fulfillments, billings, product, and settlement.
2. Create a second connection to the endpoint. Because it gets a smaller share of the endpoint's total concurrency, the secondary connection forms smaller request queues. Replace the original connection on lower-priority flows with the new one. Start small so you can isolate the variables, and test the effect.
3. Give the connection that serves high-priority flows a higher concurrency level. See Example A. The maximum concurrency level depends on your Celigo edition.
4. Repeat with a third connection or more, if needed. Adding too many low-concurrency connections to the same account can hurt performance.

4. Borrow concurrency from a main connection

Use this step when you have more than one integration app in your integrator.io account, or custom flows built inside an app. Borrowing concurrency is an advanced option that needs careful setup and repeated testing. The maximum concurrency level depends on your Celigo edition.

1. If you run several integration apps, repeat step 3 to add a connection to each one.
2. Choose one integration app's connection as the main connection.
3. Set every other integration app's connection to that endpoint to borrow concurrency from the main connection. See Example B.

Tips for working with multiple connections

• Don't replace a connection in a flow that's running. If you do, the integration app cancels all jobs for the affected flow steps.
• If a flow belongs to a chain of flows (two or more flows that trigger one after another), use the same connection for every flow in the chain.
• Use the same connection for these flow pairs. Customer sync is a low-concurrency flow and order sync is a high-concurrency flow; with different connections, the order sync fails and is blocked.
  • Order and customer sync in any eCommerce integration app.
  • Estimate and customer sync in the Salesforce - NetSuite integration app.
• In the Amazon - NetSuite integration app, run all settlement flows with the same NetSuite connection. To give a flow a secondary connection, select it in Select flows to allot secondary NetSuite connection (Settings > General section > General).
• If you clone the Salesforce - NetSuite integration app after adding connections, all the connections are cloned from the source tile to the destination tile.
• If an integration tile has multiple stores, accounts, or banks, a new connection applies to all of them.
• If you add a lookup, import, or export step to any of these flows, use the same connection for all the flow's imports and exports:
  • Shopify: Shopify Order to NetSuite Order Add, Shopify Order to NetSuite Cash Sale Add, Shopify Order to NetSuite Order/CashSale Add, and Shopify Payout Transactions to NetSuite Deposit.
  • Amazon: NetSuite Transfer Order to Amazon Inbound Shipment Plan.

Note: Uninstalling the integration app deletes all its connections. You can't delete the primary connection, even if it's unused or offline. To remove an unused secondary connection, see Add multiple connections to your integration app. Stores, accounts, and banks aren't checked against a connection, so configure each connection with the correct NetSuite account ID.

Examples for optimizing connections

Both examples follow the same general approach:

1. Create a secondary connection, for example NetSuite (Low Traffic).
2. Split the total available NetSuite concurrency: give a generous share to your default NetSuite connection and the rest to the secondary connection.
3. Replace the connection on all low-priority flows (everything except order flows) with NetSuite (Low Traffic).
4. If you have more than one integration app, borrow concurrency.

Use the data you gathered while troubleshooting to tailor the approach to your apps, data, and flows.

Example A: Single integration app, prioritizing order flows

A Shopify - NetSuite integration app starts with the default flows and connections, already at the maximum concurrency allowed for the account:

The holiday season is expected to bring 50,000 new orders and 100,000 item updates per day. With 20,000 items listed in Shopify, the extra traffic will slow the inventory, pricing, and item flows. Adding connections that favor order traffic, and especially the heavy Shopify imports, solves this:

Example B: Multiple integration apps borrowing concurrency

A customer runs four integration apps daily, all querying NetSuite often: Shopify - NetSuite, Amazon Seller Central - NetSuite, eBay - NetSuite, and Extensiv - NetSuite. The NetSuite account allows 15 concurrent requests, split across the four connections:

This setup caps every app. If Shopify is the most active and the others run less often, the Shopify flows still get only 5 concurrent requests at a time, and the concurrency in the other connections sits unused. The simplest fix is to give all the concurrency to one main connection and let the other apps borrow from it:

Note: A connection that borrows concurrency can't have its own concurrency level; that setting is disabled.

To prioritize certain flows, add secondary connections for low-priority or low-volume flows or steps, and split the concurrency into two pools, each with its own main connection:

Related articles

• Add multiple connections to your integration app
• Run flows in parallel with a secondary NetSuite connection
• Optimize integration app performance during high-volume periods
• Unlock additional fields in your integration app flows