Allow access to logs of completed imports and allow retry of data

Craig Silk

• June 01, 2020 02:20

At the moment you can only see the retry data for errors and resolved imports. Sometimes some bad configuration could lead to legitimate data being ignored. It would be good if we could retry the ignored data after changing flow config.

It's very frustrating to go back to the source of the data (third part) and request that they resend the same data again.

1

• Official comment

  

  viliandy leonardo Product Management Director

  • June 02, 2020 17:18
  • Edited

  Craig Silk 

  Thank you for submitting idea.

  Integrator.io retains error data so that users can retry the data after fixing integration issues. Success and Ignored records are not persisted by design because storing these extra data types not only leads to higher operational cost (in terms of storage extra capacity and data protection) but also is not a feature required by all of our customers; some prefer that io does not store any non-error data. 

  Integrator.io is a flexible and powerful integration platform that has tool to build what you need here. You can add an import step to persist the ignored records in your own data store (i.e S3 bucket, FTP, or a Database).  Here is an example that shows an import step to store ignored records in a S3 data store. That said, I recommend that you improve the existing filter to catch all legitimate data before considering storing ignored records.

  

Please sign in to leave a comment.