This article provides a conceptual overview of webhook technology as it relates to integrator.io and includes the steps necessary to create listeners in your flow.
Contents
Understanding webhooks
Webhooks send an HTTP POST message over the web in real-time to an endpoint when an event occurs in an application. When an event triggers a webhook request, the source application sends a message to a specified URL, and a webhook listener receives the message and then executes operations according to the message's content.
Why use webhooks?
Before webhook technology was available, if you wanted to sync information in two places across the web, one system would have to send repeating poll messages to review the status of the data in the system it needed to retrieve new information from, and then update data based on the responses to the poll requests. Webhook technology eliminates the necessity for constant polling by allowing one system to automatically send a notification and all relevant information of a given change to another system as soon as the change occurs.
Webhooks in integrator.io
integrator.io supports webhook technology on source application exports. If you set up webhook settings within the user interface of your source application, you can configure listeners in integrator.io to receive messages from the source app and trigger integration flows based on the message content. Setting up webhooks for integrator.io requires two steps:
- Configure a webhook in the source application for an event that triggers the HTTP POST message from the source application to integrator.io.
- Create a webhook listener in integrator.io to receive incoming information from the source application and initiate the flow with the data sent.
Webhook listener content types
integrator.io webhook listeners can receive the following content types:
- application/json
- */xml
- application/x-www-form-urlencoded
Webhook listeners can't parse CSV data or any other content types besides those listed above.
Tip: Organizations may use different terms for webhook technology. Consult the API documentation for your source app to determine how it refers to these concepts.
Create a webhook listener
These instructions apply to the universal Webhook connection. If you are listening for a webhook from a built-in integrator.io connection, instead choose that Application and then configure the export to use a webhook.
- From the Resources menu, select Export.
- In the resulting Create export panel, select Webhook for the Application setting.The Create listener panel opens.
- Enter a Name and a brief Description for the webhook.
- Click Next.
Secure the listener
Choose the verification type that will provide security for the webhook listener.
Basic
The simplest verification method, relying on transport-level encryption for each request.
Enter your Username and Password.
HMAC
A hash function and a secret key authenticate requests, configured with the following settings:
- Algorithm: SHA-1, SHA-256, SHA-512, or SHA-384
- Encoding: Base64 or Hexadecimal
- Key (secret): This is the secret key provided to you for the webhook. This is required to generate the hash value.
- Header (containing hmac): This is the header name in your HTTP request. For example, the header name for Slack webhooks is "X-Slack-Signature". To get a signature, encode the request body using the specified algorithm and secret.
Secret URL
Provide a generated token and a secret temporary URL to authenticate requests in the exposed settings.
Token
Authenticate token-based requests with a provided or generated token.
Token (required): Generate a new token or enter the token that will be sent by the webhook origin so that integrator.io can authenticate the request to receive data from that endpoint. Only share this token with the webhook provider.
Path (required): Enter the JSON path of the field, such as myAuth.token, in the request body that will contain the token value.
General URL & sample data
Public URL (required): If a URL has not been generated yet, click the plus icon (+) to generate a public URL for this webhook listener. You can then share this URL with the webhook provider by entering it in the provider's user interface or API. Each webhook provider has a methodology to configure a destination URL where the webhook will send data, so use the public URL as the destination URL when you are configuring the webhook in the host application.
Sample data: Click Click to show to populate the sample data field with the webhook data most recently received by the public URL endpoint. If the public URL endpoint has not yet received any data, you can manually enter sample JSON data in the field provided.
Advanced
Open the Advanced section for additional webhook configuration options.
Override HTTP status code for the success responses: integrator.io returns a 204 HTTP status code for all successful webhook requests by default. Select an alternate successful HTTP status code for the listener.
Override media type for success responses: JSON is the default media type for success responses. Select an alternate media type from this list.
Override HTTP response body for success responses: integrator.io returns an empty response body message for all successful webhook requests by default. Use this field to customize an HTTP response body returned for successful requests made to the listener. Copy and paste the data you sent to integrator. io into the input field to configure a handlebars expression.
Page size: When an export runs in the context of a data flow (where the data from the export is sent right away to an import queue) integrator.io will break the data being exported into one or more smaller pages of records. Saying this another way, integrator.io uses streaming to export data out of one app and import it into another app. The Page size field allows you to specify how many records you want in each page of data. If you leave this field blank, the default page size is 20 records. There is no maximum record limit, but a page of data is limited to 5 MB. Most of the time, the application that you are importing data into will bottleneck the page size value. For example, if you are importing data into NetSuite or Salesforce they each specify (in their API guides) a maximum number of records that can be submitted in any single request.
Data URI template: When your flow runs but has data errors this field can be really helpful in that it allows you to make sure that all the errors in your job dashboard have a link to the original data in the export application. This field uses a handlebars template to generate the dynamic links based on the data being exported. For example, if you are exporting a customer record from Shopify, you would most likely set this field to the following value https://your-store.myshopify.com/admin/customers/{{{id}}}. Or, if you are just exporting a CSV file from an FTP site then this field could simply be one or more columns from the file: {{{internal_id}}, {{{email}}}, etc...
Do not store retry data: Check this box if you do NOT want integrator.io to store retry data for records that fail in your flow. Storing retry data can slow down your flow's overall performance if you are processing very large numbers of records that are failing. Storing retry data also means that anyone with access to your flow's dashboard can see the retry data in clear text.
Override trace key template: Define a trace key that integrator.io will use to identify a unique record. You can use a single field such as {{{field1}}} or use a handlebar expression. For example, this syntax {{join “_” field1 field2}} will generate a trace key template field1_field2. When this field is set, you will override the platform default trace key field.
Webhook listener debug logs
Rate this feature
Included in the June 2021 release integrator.io allows you to capture requests received by a webhook listener within a specified time period. You can then view the logs to verify that the data includes all expected information.
Note: See debug flow steps for information on debug logs for other flow step types.
Use the following steps to review data sent from a webhook:
- Open the flow that contains a listener export step.
- Click Listener to open the Edit listener panel.
- Click View debug logs in the upper right corner of the Edit listener panel.
- Click Start debug and choose a period of time for the debugger to capture data.
- Perform an action that generates the trigger to send data to the webhook listener within the external application.
- Select a log to review data received by the integrator.io webhook listener.
Webhook listener debug logs demonstration video
Comments
8 comments
What content types will the Integrator webhook accept? Will it accept multipart/form-data?
Hi, Shane Brown. Thanks for the question, which isn't directly addressed in this article yet.
And sorry for the delay. I needed to track down the answer as well as the relative priority for adding this feature to integrator.io.
Webhooks that receive multipart/form-data are not yet supported in the platform, but we are looking at adding this format, hopefully within the year.
How many concurrent requests can a webhook handle? we are looking to have a single webhook listener exposed for a number of partners, but we would like to understand its capacity, can you please provide some details about it? thanks.
Is there a method to define a Resource Path if we are using a Webhook Listener with XML?
XML Structure is as follows and we want to make sure this is send as 2 records. If we cannot define a path of /Shipments, then this is seen as one record.
<shipments>
<shipment>
.....
</shipment>
<shipment>
.....
</shipment>
</shipments>
Hello Martin Jahn,
Our webhook exports don't yet support the selection of specific fields via resource path, but in your import step, you can access any specific field returned from the webhook export. Please let us know if you have any trouble with that solution.
Tom
As I understood, the WebHook will immediately return a response to the source app upon receiving the payload. Is there any way we can delay sending the response until the flow has been completed?
Basically, I would like to send a 200 response if the flow ran without any errors and send a 4XX/5XX response if there were any errors in the flow.
Hi Zanfer Hussain,
I'm having a little trouble understanding your request. A webhook kicks off the flow with the source data sent from the export. At that point the data begins passing through your flow. I'm not sure what you're attempting by delaying the response. Could you add more clarity?
What's the request limit a webhook can handle? If we're sending thousands of requests per second to a webhook listener, is it better to create multiple workflows?
Please sign in to leave a comment.