Streamlining Netsuite lookup

Hello, we have a very large file that we use for price updates (about 1152900 lines of data) 

My flow currently is google drive > Netsuite lookup > import into netsuite

The netsuite lookup is used to determine if we have it in our catalog before importing the item price data. This seems to be a bottle neck and is very slow (I assume because of the sheer number of items its looking up)

Is there any way to make my flow more efficient? I have not yet found anything online

0

Comments

3 comments
Date Votes
  • Hi Matthew Gargano,

     

    I have a couple questions about your setup first.

    1. On you NetSuite import step, are you updating an item record and then map to the item pricing sublist?
    2. On your Google Drive step, are you using the group by functionality to group different item prices to each SKU?
    3. What is the concurrency setting on your NetSuite connection within Celigo? You can find this my editing the connection and going under the bottom advanced section.
    4. How many items from the file don't actually live in your NetSuite environment (i.e. how many items from the file are you discarding)?

     

    The reason I ask is because I would setup the flow like this:

    1. On the google drive step, group your data based on SKU.
    2. Get rid of the lookup step.
    3. Remake the NetSuite import step and choose "update" as the operation and check "ignore missing record" and fill in the "how can we find existing records" AND under advanced choose SuiteApp 1.0. You can't choose SuiteApp 1.0 after saving the import step so you need to remake it and choose it prior to saving. The reason to use SuiteApp 1.0 is due to a bug that should be fixed here in a couple weeks.
    4. Now map in header fields if needed and map in your pricing fields to the price sublist.
    5. Edit your NetSuite concurrency up to a comfortable level relative to your overall NetSuite concurrency limit.

     

    The reason for getting rid of the lookup is because it's really just doubling the number of lookups since the import step is already doing that. When you choose "update" on the import step, it's already performing a "lookup" behind the scenes to find that record. However, if half the file is just SKUs you don't even need, then how you have it makes sense. By adding grouping, there are fewer records passing back and forth and you can bulk update pricing per item as opposed to one item/price combo at a time.

     

    Hopefully that helps

    0
  • Thank you for all of that info. 

    To answer your questions:

    1. The only thing being updated is the items vendor price and a date field.

    2.Each line on the file is a different item, so I dont think the grouping is relevant. unless I misunderstood what you meant.

    3. I have to look more into the concurrency, currently I have that blank. I dont anticipate that there will be any duplicate items in the file

    I have cloned the flow, removed the lookup and reworked the mapping. I am currently testing

    Thanks again

     

    0
  • Matthew Gargano that makes sense. I was assuming you were updating multiple price levels per item. For example, if you had a Walmart price level and an Amazon price level for the same item that you needed to update, you could use the grouping feature to group those 2 rows together and make 1 call to NetSuite to update both at the same time. In your case, it sounds like you are just updating a header level value on the item record so there wouldn't be a need to group the data.

     

    As for concurrency, here are a few good articles. If it's left blank, it will use a concurrency of 1. I would suggest increasing it.

    https://docs.celigo.com/hc/en-us/articles/360034522472-Govern-concurrency-of-NetSuite-connections

    https://docs.celigo.com/hc/en-us/articles/360043926372-Configure-connections-to-optimize-throughput-and-governance

    https://docs.celigo.com/hc/en-us/articles/360043927292-Fine-tune-integrator-io-for-optimal-performance-and-data-throughput

    0

Please sign in to leave a comment.

 

Didn't find what you were looking for?

New post