Using exports.runVirtualWithPaging

bill budwitz

• June 16, 2022 13:48

I am trying to implement exports.runVirtualWithPaging and have a question regarding getting the next page(s).  Based upon the explanation of how to use it, i am passing back the captured pagedExportState object retrieved from the first call.   The return on the second call is always an empty array (no data).  The first call does successfully return 25 records and the pagedExportState array.

Here is an example of the pagedExportState object i retrieved from the first call:

{

    "pageSize": 25,

    "recordCount": 25,

    "pageIndex": 1,

    "done": true,

    "issuedAt": 1655387043,

    "sig": "b96c490939ceee5a04056353c707961d7187ff7bd8f09ccf255429c33b53fb51"

}

Here is the code i am using.  Any help around the payload structure for the callback would be appreciated.

import { exports } from 'integrator-api'

function _mainExport (options) {
  let response = {};

    try {
      let exportObj = productsExportObj ();
     

//initial call
      let responsePage1 = exports.runVirtualWithPaging({export:exportObj});
      let pagedExportStateObj = responsePage1.pagedExportState;
      

//callback
      let responsePage2 = exports.runVirtualWithPaging({export:exportObj, pagedExportState:pagedExportStateObj});
      

//return full response from callback
      response.body = responsePage2;
      response.statusCode = 200;
    } 
    catch(e) {
      response.body = JSON.stringify(e);
      response.statusCode = 400;
    }
  

  return  response;
}

function productsExportObj () {

  let exportObj = {
  "_connectionId": "615xxxxxxx24823", 

   "apiIdentifier": "ecxxxxxxda",

    "asynchronous": true,
    "oneToMany": false,
    "parsers": [],
    "http": {
      "relativeURI": "apikey/products",
      "method": "GET",
      "successMediaType": "json",
      "errorMediaType": "json",
      "formType": "rest"
    },
    "adaptorType": "HTTPExport"
  }

  return exportObj;
}

0

• 

  Bhavik Shah Senior Product Manager

  • June 17, 2022 11:12

  

  

  bill budwitz

  I don't see any issue with your code. Perhaps this appears to be a data issue. Are you sure there are enough records to be fetched for 2nd page ? 

  0
• 

  bill budwitz

  • June 17, 2022 13:39

  

  

  Bhavik Shah there are over 600 records at that endpoint (Postman and a celigo import flow returns this many).   There is another issue that Celigo is "working" on that will return the number of records (pageSize) return on an exports.run call.  currently it defaults to 25 max and no matter what pageSize is set too it still just returns 25 records.  Maybe related, i dont know.  very frustrating.  

  0
• 

  Scott Henderson CTO

  • June 18, 2022 19:23

  

  

  

  Hey Bill,

  Where is your script running?  Is it in a 'MyAPI', or is it running in a 'hook' in a flow?  I am asking because if you are doing this in a MyAPI, and this is the only thing the MyAPI function needs to do, then we also have a standard REST API that you could call that would allow you to do the exact same use case, but without having to write any JavaScript.  See here, and look for the following sections.

  

  We will work on the Celigo side to fix any bugs asap, but hopefully we can also find you a work around in the meantime.

  0
• 

  Bhavik Shah Senior Product Manager

  • June 21, 2022 16:55

  

  

  bill budwitz We got this checked with our internal team. It appears that paging method properties are not part of the export object which is why system is returning default values. Could you please add paging properties related parameters in export object and check if that works ? 

  Eg : pagingMethod: 'nextpageurl', nextPagePath: 'nextPageURL'
  
  Meanwhile we will update help docs to ensure these nuances are captured. 

  0

Please sign in to leave a comment.