Data displayed in the preview is outdated
Looking at the preview window available in many panels, I'm pretty sure now that the data displayed is cached. The problem with that is the data is very often outdated. This makes it so that developing mapping and scripts to handle multiple scenarios is very difficult as the preview only displays one version of the data.
Is there a way to "force updated" the preview data? It seems to be inconsistently updating nowadays.
I edit the first export of the flow, refresh the preview data and then change the description field and save the export. This seems to update the cache. I have no idea if this is real or imagined, but it seems to work.
Is there an official comment from celigo on this topic?0
Hi David Gollom,
Your procedure worked :) I was pretty sure there was a way to trigger a cache refresh since the issue was inconsistently happening. Thanks a lot !
+1 in requesting either a fix or a feature to force a cache refresh
To provide more detail, we will try adding the random handlebar function into a header param inside our export. Hence, the cache should always be renewed. Pretty wild workaround as I assume this cache is not there for no reason but when push comes to shove :)1
Hi Philippe Foisy,
Let me know if that random helper works succesfully.
Hello David Gollom Philippe Foisy,
Great to see the collaboration on this thread. We're trying to solve this problem by allowing users to add mock data in exports. When provided, mock data could be used to configure the flow. This would also help in situations where the preview data does not contain all the fields required to configure the flow. Do follow notifications for upcoming releases for an update on this change.
Hi David Gollom and Philippe Foisy,
It is not your imagination :) - We cache the preview data until there's a change in the upstream exports. If there has been no change upstream, then the input to the flow step you're currently working with wouldn't have changed and hence you see the same cached data. If you make a change to an upstream flow step so that the data is updated, then the current flow step input would be changed and hence the preview refreshed. Hope that makes sense.
What Amanjot Singh is referring to, which will be of great help in your flow configuration, is this feature which the team is currently working on, targeting 1H of 2023.
Hope this helps!
Thank you Katarina Brunnestom and Amanjot Singh for your replies. I'm glad to know that my little procedure wasn't just the placebo effect.
I'll disagree with something said. When creating NetSuite exports, the data can change without the export changing. This happens all the time. I'll change a saved search that provides data to the export. The export doesn't change but the data does. I can add new columns to the search.
I suggest a feature for connectors like NetSuite, where the data being exported can change like this, have so indicator that doesn't cache the data.
Thanks for your attention to this issue!
Thank you Katarina Brunnestom and Amanjot Singh for this helpful feedback
I gotta concur with David Gollom here. We are integrating with Adobe Commerce, Shopify, WooCommerce and Google Merchant at the moment and it happens that the data returned by an API request will differ depending on the context of each record.
Since we are doing eCommerce 2 eCommerce integrations, most of our challenges are surrounding types of products/orders that are defined differently. A single API request will very often return multiple types of products/orders. Since scripts are only "event aware" we got to make the scripts "data aware" so they handle different types of entities differently. That's where the preview fails us all the time :)0
The upcoming feature we are planning for early next year will let you define the exact "mock output" data that you want to be stored/cached for each export bubble in your flow. We will make it really easy to refresh the mock output data stored on the export by making actual preview calls on the fly. Most importantly, the mock output data will allow you to lock down the perfect payload that has all the different records and data types that your flow needs to process (i.e. you can define multiple records in the mock output), so that you don't need to constantly be bouncing back and forth testing with only a single record at a time, and then retesting flows in the future will obviously be much easier when you have these test payloads already built out and always available, etc...0
That's golden ! Are you targeting the release of H1 2023 ?
Thanks Scott Henderson those new features sound great.0
Yes, sometime in H1.0
Please sign in to leave a comment.