How do I separate the file per process using custom flow FTP to FTP?
Hi, I encountering a minor issue wherein I have to parse 2 or more XML files from FTP going to FTP but in JSON format. Once I process the files, it keeps merging the record and giving me 1 file as a result. What I want to achieve is if I have a 2 or more files. I should expect 2 or more files also as a result after the process.
Thank you!
0
Comments
Mario A. Belango Jr are the XML file names different? If so, you can just have another export on your flow since each export creates it's own file. Just use the file filter options to get just the file you need.
Yes the filenames are different, but the file name don't have a specific format. I already tried adjusting batch, skip aggregation and page configuration but still getting the same result
Mario A. Belango Jr I'll say there isn't a great solution here, but I'll give you one that works if your files are under 5 MB each.
If your file is under 5 MB, then:
function preSavePage (options) {
if(options.data && options.data.length>0){
for (let i = 0; i < options.data.length; i++) {
options.data[i].fileName=options.files[i].fileMeta.fileName;
options.data[i].fileSize=options.files[i].fileMeta.fileSize;
}
}
return {
data: options.data,
errors: options.errors,
abort: false,
newErrorsAndRetryData: []
}
}
Mario A. Belango Jr here is a zip file you can use as well for what I made.
https://drive.google.com/file/d/1smU15NxItfyv7MDCOZfccMKzCl5CbN1o/view?usp=sharing
Thanks for this Tyler, I will give a feedback once I tried this approach.
Hi Tyler Lamparter, have you tried setting the page to 1 then try to check the skip aggregation? So you only need two flow instead of 3? Is there a pros and cons using the approach you did vs the configuration I did? Your input is appreciated. Thanks!
Mario A. Belango Jr you could do that, but then you would end up with 1 JSON file per page.
Hi Tyler, can you elaborate it more what do you mean by 1 JSON file per page? I just want to understand it further. Because after I configured the 1 page then skip aggregation. I finally achieved the result, where in I process 3 files then get 3 files as a result.
Mario A. Belango Jr yeah it all depends on your setup. Here are some scenarios below:
Hopefully that clears it up.
Please sign in to leave a comment.