In a rare situation, you might need to extract the information from the error retry data for further research. Assuming you are working with a custom flow, one way to automate the process is to write a postSubmit hook to send the payload info to a listener in a real-time flow.
Consider the following example:
Add a JavaScript hook to an import step
Select a script file or start a new one (+); specify the entry-point function
The following sample code sends the response data to an integrator.io listener (webhook) for further processing, only when the response is not successful:
/*
* postSubmitFunction stub:
*
* The name of the function can be changed to anything you like.
*
* The function will be passed one ‘options’ argument that has the following fields:
* 'preMapData' - an array of records representing the page of data before it was mapped.
* A record can be an object {} or array [] depending on the data source.
* 'postMapData' - an array of records representing the page of data after it was mapped.
* A record can be an object {} or array [] depending on the data source.
* 'responseData' - an array of responses for the page of data that was submitted to the
* import application. An individual response will have the following
* fields:
* 'statusCode' - 200 is a success. 422 is a data error. 403 means the connection went
* offline.
* 'errors' - [{code: '', message: '', source: ‘’}]
* 'ignored' - true if the record was filtered/skipped, false otherwise.
* 'id' - the id from the import application response.
* '_json' - the complete response data from the import application.
* 'dataURI' - if possible, a URI for the data in the import application (populated
* only for errored records).
* '_importId' - the _importId currently running.
* '_connectionId' - the _connectionId currently running.
* '_flowId' - the _flowId currently running.
* '_integrationId' - the _integrationId currently running.
* 'job' - the job currently running.
* 'settings' - all custom settings in scope for the import currently running.
*
* The function needs to return the responseData array provided by options.responseData.
* The length of the responseData array MUST remain unchanged. Elements within the
* responseData array can be modified to enhance error messages, modify the complete
* _json response data, etc.
* Throwing an exception will fail the entire page of records.
*/
import { exports } from 'integrator-api'
function postSubmit (options) {
// Response data contains error
if (options.responseData.statusCode != '200') {
/* Send data to a listener */
exports.run({_id: '••••••••••••••••', listenerData: options.responseData})
}
return options.responseData
}
Example flow with webhook step to receive errors sent via the hook; branched destinations to upload the error and send a message to the team channel
It is important to note that the response data will not contain an error ID that you can use to manage the error automatically when attempting to retry or resolve the error.
Comments
Is it possible to send XML data to an integrator.io listener and have the XML parsed?
The errors from the connection I'm trying to send to the integrator.io listener are XML. I'm looping over the errors, getting the XML, and passing it to the integrator.io listener through the listenerData field, like above.
However, the listener Flow returns
{ “data”: “<myxmlstring>” }as the data to work with which isn't super useful. I need the XML parsed out into JSON so I can concat the error messages accordingly.NOTE: I did try to manually parse the XML in a script file using methods from https://developer.mozilla.org/en-US/docs/Web/XML/Parsing_and_serializing_XML but it seems
DOMParseris not allowed in celigo scripts.I also didn't see anything on the connection to tell it to use XML and https://docs.celigo.com/hc/en-us/articles/20239758683035-integrator-io-API-and-JavaScript-runtime-objects#export-runtime-objects-1 doesn't indicate any settings/parameters you can pass in to tell it the content is XML.
Josh Braun it looks like XML isn't supported in the JSRT functions, but the IO listener does accept XML data to it. So as a work around, you could:
Since IO listeners do accept XML, but not from scripts, the second flow here is essentially converting a script call to a normal import bubble call.
Tyler Lamparter
Thanks for the reply.
So basically a middle-man flow that takes the JSON with XML and passes the XML along in the body directly of another Flow that actually exports the data to where ever it needs to go?
Josh Braun correct. It's not the prettiest, but should get the job done.
FYI: we've added these enhancements to the backlog of ideas (support XML for JSRT functions and allow other JS libraries that could support parsing directly in script).
Gotcha, makes sense.
What is the best way to do the import to an IO listener? Use the IO connection or use HTTP and just set the URL to the target IO listener?
And good to know, thank you. Being able to set the content type of the exported data would be useful, that way you have control how it will be sent.
EDIT:
When I look at using an IO connection it looks like it wants me to call an API, and none of the options are call/run. However, using HTTP I'm getting unauthorized but I'm not 100% sure how the authentication should be setup for calling the URL for an IO listener. Is an API Token needed and set as Token auth?
EDIT EDIT:
API Token set as the Bearer token worked.
Josh Braun yeah you can use HTTP connector or the integrator.io connector. In the post body of the import step, you would just put something like
{{{record.xml}}}where the xml field is the xml text field you have in your JSON record.I created the Celigo Listener and used the export ID there to call it from the Erroring Flow postSubmit hook. However, when this is triggered, I am resulting to the following script_error in the Erroring Flow:
Message:
Access restricted
Code: script_error
Source: Post submit hook
Timestamp: xxxxxxxx
Error ID: xxxxxxxx
Trace key : xxxxx
Any idea why this happens?
Neigel Yap you're probably in standalone flows. This only works when in a dedicated integration tile and not in standalone flows.
Please sign in to leave a comment.