Using exports.runVirtualWithPaging
I am trying to implement exports.runVirtualWithPaging and have a question regarding getting the next page(s). Based upon the explanation of how to use it, i am passing back the captured pagedExportState object retrieved from the first call. The return on the second call is always an empty array (no data). The first call does successfully return 25 records and the pagedExportState array.
Here is an example of the pagedExportState object i retrieved from the first call:
Here is the code i am using. Any help around the payload structure for the callback would be appreciated.
import { exports } from 'integrator-api'
function _mainExport (options) {
let response = {};
try {
let exportObj = productsExportObj ();
//initial call
let responsePage1 = exports.runVirtualWithPaging({export:exportObj});
let pagedExportStateObj = responsePage1.pagedExportState;
//callback
let responsePage2 = exports.runVirtualWithPaging({export:exportObj, pagedExportState:pagedExportStateObj});
//return full response from callback
response.body = responsePage2;
response.statusCode = 200;
}
catch(e) {
response.body = JSON.stringify(e);
response.statusCode = 400;
}
return response;
}
function productsExportObj () {
let exportObj = {
"_connectionId": "615xxxxxxx24823",
"apiIdentifier": "ecxxxxxxda",
"asynchronous": true,
"oneToMany": false,
"parsers": [],
"http": {
"relativeURI": "apikey/products",
"method": "GET",
"successMediaType": "json",
"errorMediaType": "json",
"formType": "rest"
},
"adaptorType": "HTTPExport"
}
return exportObj;
}
Comments
bill budwitz
I don't see any issue with your code. Perhaps this appears to be a data issue. Are you sure there are enough records to be fetched for 2nd page ?
Bhavik Shah there are over 600 records at that endpoint (Postman and a celigo import flow returns this many). There is another issue that Celigo is "working" on that will return the number of records (pageSize) return on an exports.run call. currently it defaults to 25 max and no matter what pageSize is set too it still just returns 25 records. Maybe related, i dont know. very frustrating.
Hey Bill,
Where is your script running? Is it in a 'MyAPI', or is it running in a 'hook' in a flow? I am asking because if you are doing this in a MyAPI, and this is the only thing the MyAPI function needs to do, then we also have a standard REST API that you could call that would allow you to do the exact same use case, but without having to write any JavaScript. See here, and look for the following sections.
We will work on the Celigo side to fix any bugs asap, but hopefully we can also find you a work around in the meantime.
bill budwitz We got this checked with our internal team. It appears that paging method properties are not part of the export object which is why system is returning default values. Could you please add paging properties related parameters in export object and check if that works ?
Eg : pagingMethod: 'nextpageurl', nextPagePath: 'nextPageURL'
Meanwhile we will update help docs to ensure these nuances are captured.
Please sign in to leave a comment.