Handlebar trim messes up import mapping output (REST API)
Hi,
I have the following issue with a import mapping in a custom data flow:
Base is an export from NetSuite which delivers order lines, which are grouped. So we have following input data (simplified):
[
{
"id": "261209",
"recordType": "salesorder",
"internalid": "261209",
"orderNumber": "S-003802",
"amount": "2",
"item description": "Baselayer Shorts M",
"sku": "4711-AAA"
},
{
"id": "261209",
"recordType": "salesorder",
"internalid": "261209",
"orderNumber": "S-003802",
"amount": "3",
"item description": "Baselayer Shorts L",
"sku": "4711-BBB"
}
]
An the following mapping:
Output is looking like expected - new sub-object "shipmentItems" with correct values for SKU, amount and desc.
Cause of data qualitiy issues we have from time to time whitespaces at beginning or end of SKU - so we need a trim-Handlebar to remove these.
But as soon as we use {{trim sku}} for SKU the whole output is messed up and SKU of the first record is used for all other sub-records.
Did not expect this behaviour - is this a bug or works as designed?
Thanks
Stefan
Comments
When you do your own expression, you need to tell the mapper it's looking at an array-to-array mapping. In your mapping for instance, IIO is really using sometime like [*].amount. The fact that there's an array on the left side of the expression is being hidden from you. However, I don't think that literal syntax will work.
I always use an export transformation to make the lines into an array nested within the json object. Then you can use something like {{trim lines[*].sku }} as your handlebar expression.
Would the $[*] syntax work in this example??
Hi David,
thanks for your feedback - I have tested a couple of things within the mapper - following expressions are NOT working:
{{trim [*].sku}}
{{trim $[*].sku}}
Interestingly this is working correct (btw. never seen this kind of expression before):
{{trim *.sku}}
But you are right, creating a transformation on export side would be the best solution here. :)
Just wondering, why it's not mentioned in the documentation, that "some magic" is happening in the background for basic array-to-array mappings (without handlebars)...
Thanks
Stefan
My $0.02 is that when the editor tries to simplify or hide details, it ends up being more of a liability than a help. I would prefer to see the full path/syntax of the data.
Stefan Lindner You could solve this problem easily using mapper 2.0. It's available using the toggle at the top for HTTP/REST imports right now. This way you wouldn't have to create an additional transform in export just to handle arrays. have added the mappings example below for your use case.
Let me know for any further questions.
To add a little color to the post from Aman, we put a lot of thought into how mapper 2.0 should work with all the different types of data. Mapper 2.0 is a very different mapping engine than 1.0, and there will be a learning curve, but once you get the basics, it will be much easier to map objects, arrays, or any other complex data structures you can imagine.
Here are some key changes:
Hi all,
thanks for your very helpful feedback - appreciate that!
It was an existing flow from pre-"Mapper 2.0" area, so I don't want to re-do the whole mapping only for a "simple" add of trim handlebar. But it was an expensive lesson learned, that it's not so easy as it looks like for array-to-array mappings with Mapper 1.0 and handlebars... (maybe it's worth to document this in any way for all other users?)
To sum it up, recommendation from Celigo is to use for every new flow Mapper 2.0, even it is in "Beta" status at the moment?
Thanks
Stefan
PS: Is there any way to mark this thread as resolved? ;)
Thanks, Stefan. I've marked the post as "Answered".
Please sign in to leave a comment.