Optimize throughput & governance by configuring connections’ concurrency levels

Comments

6 comments

  • Anna Lam

    For example, if the concurrency level is set to 6 and a long-running export is running in one of your flows, then you will still have 5 messages that can be processed in parallel. Those additional 5 messages could also be long-running exports, or they could be very quick record imports. It does not matter.

    In this example, could those additional 5 messages be from the same long-running export? Or does it need to be from a different export?

    If we are running only one export with a concurrency level of 6, does that mean up to 6 records from that same export can be processed concurrently? If so, what happens when another export using the same connection suddenly starts? How do those two exports split the concurrency levels?

    0
  • Tyler Lamparter Principal Product Manager
    Awesome Follow-up
    Engaged
    Top Contributor
    Answer Pro
    Celigo University Level 4: Legendary

    A long running export (if on the source side of the flow) will typically only use 1 concurrency because it's going out and making 1 api request for a page of data. The concurrency level is for how many api requests we are making to the endpoint, which can be different than the number of actual records when getting a list of records back from an api or making a batch import request to an api. 

    In the case of a long export, we would make one request to get the first page of results, then make subsequent requests for more pages of data. This inherently uses 1 concurrency because I have to get a response back before making a request for a new page of data.

    Where an export could take up more concurrency is if the export is a lookup step within your flow. For example, if you need to make an api request per record in order to get additional information about the original record. In this case, the lookup step would use additional concurrency because it can ping the api in parallel for each specific record.

    Hopefully that helps! 

    0
  • Anna Lam

    Thanks Tyler. Do you have any insight on how we can improve processing time for single long running imports? We are using the Amazon <> Netsuite IA and we occasionally have long processing times for the Netsuite import step. It sounds like adding concurrency does not improve the processing time for a single flow but it would allow more than one flow to process at a time. Is there a way to reduce the processing time for a single long running import?

    0
  • Tyler Lamparter Principal Product Manager
    Awesome Follow-up
    Engaged
    Top Contributor
    Answer Pro
    Celigo University Level 4: Legendary

    Hi Anna - what NetSuite imports are you seeing long processing times for?

    Also, maybe I need to reword my response here. Increasing your concurrency definitely could have benefits for your import steps (an import step is where we are importing data into the final destination; export step is where we are fetching data from a source and bringing it into Celigo). When the import step is creating or updating on a per record basis, increasing the concurrency allows multiple records to be created or updated in parallel. For example, if you are pulling orders from Amazon and importing them to NetSuite, if your NetSuite concurrency is set to 1, then we will create/update 1 order in NetSuite at a time. So we will make 1 request to NetSuite, get a response, then make an additional request for the next order. If you were to set your NetSuite concurrency to 2 on the NetSuite connector, we would send 2 requests to NetSuite at the same time in order to create/update 2 orders at one time in parallel. 

    In your initial question, you asked about exports and how concurrency was used there. For example, if I have an export to get Amazon orders, I make a single api request to get 50 orders at a time. Subsequently, once I get the response for the first 50 orders, I then make another request for the next 50 orders and so on and so on until I've gotten all my orders. For exports, we have to get one page of 50 records at a time before we make another call for the next page of 50. Therefore we typically only use a concurrency of 1 since we can only make 1 request at a time.

    To bring it all together, your concurrency level on the connection level is what is used across all your flows that use that connection. For example, if I have flow 1 that imports Amazon orders to NetSuite, flow 2 that is pulling inventory from NetSuite to send to Amazon, and both of those flows use the same NetSuite connection, each flow shares the concurrency of that connection.

    Does that help clarify it a bit further?

    0
  • Anna Lam

    Hi Tyler,

    That helps to clarify things a lot, thank you so much. 

    For example, if I have flow 1 that imports Amazon orders to NetSuite, flow 2 that is pulling inventory from NetSuite to send to Amazon, and both of those flows use the same NetSuite connection, each flow shares the concurrency of that connection

    In this example, say that this Netsuite connection has a concurrency of 2. Flow #1 is importing Amazon orders to Netsuite and it is the only flow running on that Netsuite connection, so it is importing 2 records in parallel by using up both concurrencies. Then say that Flow #2 starts running, does it queue up after Flow #1 or does Flow #1 lower the concurrency usage so that Flow #2 can run in parallel?

    0
  • Scott Henderson CTO
    Celigo University Level 1: Skilled
    Answer Pro
    Top Contributor

    In your example, flow 2 will queue up after the messages that flow 1 has already put in the FIFO queue to process.  If flow 1 is still exporting new source records after flow 2 starts queuing messages at the same time, then you will have both flow 1 and flow 2 messages being processed at the same time in FIFO order.

    0

Please sign in to leave a comment.