The Celigo platform supports webhook technology as a source application. If you set up webhook settings within the administrative interface, or control panel, of your source application, you can configure listeners in the Celigo platform to receive messages and trigger integration flows based on that content. Setting up webhooks requires two steps:
-
Configure a webhook in the source application for an event that triggers the HTTP POST or PUT message from the source application to the Celigo platform.
-
Create a webhook listener in the Celigo platform to receive incoming information from the source application and initiate the flow with the data sent.
Note
The following instructions apply to the universal Webhook connection. (If you are listening for a webhook from a prebuilt application, instead select that Application and for the Step type choose Listen for real-time data from source application.)
-
From the Resources menu, select Export.
-
In the resulting Create export panel, select Webhook for the Application setting. The Create listener panel opens.
-
Enter a Name and a brief Description for the webhook.
-
Click Next.
Public URL (required): If a URL has not been generated yet, click the plus icon (+) to generate a public URL for this webhook listener. You can then share this URL with the webhook provider by entering it in the provider’s user interface or API. Each webhook provider has a methodology to configure a destination URL where the webhook will send data, so use the public URL as the destination URL when you are configuring the webhook in the host application.
Sample data: Populate this field with sample webhook data. You may be able to find sample data in your application’s API documentation.
Tip
XML is supported for webhooks; however, the sample data in the webhook is limited to JSON. You can convert XML to JSON in Build > Playground with the XML parser helper.
Sample headers: Enter request headers that mimic what your webhook will receive. Use JSON in canonical format: an array of name-value objects.
Sample query parameters: Enter query parameters that are typical of what your webhook will receive. Use JSON in canonical format: an array of name-value objects.
Choose the verification type that will provide security for the webhook listener.
A hash function and a secret key authenticate requests, configured with the following settings:
Algorithm: Choose SHA-1, SHA-256, SHA-512, or SHA-384
Encoding: Choose Base64 or Hexadecimal
Key (secret): This is the secret key provided to you for the webhook. It is required to generate the hash value.
Header (containing hmac): This is the header name in your HTTP request. For example, the header name for Slack webhooks is X-Slack-Signature. To get a signature, encode the request body using the specified algorithm and secret.
Click to open the Advanced field editor (AFE) and build a handlebars template. Some SaaS vendors, such as GitHub and Shopify, decorate their signature headers with algorithm identifiers (
sha1=, sha256=), which would make HMAC validation fail without string manipulation. The AFE gives you access to the request headers and connection/flow settings for flexibility in removing any prefix or values other than the HMAC signature itself.
See Configure challenge requests for more on using a webhook key when generating a cryptographic challenge response.
Authenticate token-based requests with a provided or generated token.
Token (required): Generate a new token or enter the token that will be sent by the webhook origin so that the Celigo platform can authenticate the request to receive data from that endpoint. Only share this token with the webhook provider.
Token location in webhook request (required): Select where in the webhook request the token will be sent. Choose one of the following options:
-
Webhook request body to provide a JSON path to the token
-
Webhook request headers to specify a header name and scheme
-
Webhook request query parameters to specify a query parameter name
You will be asked to provide additional, unique settings based on this selection.
Path to token field in the webhook request body (required): Enter the JSON path of the field, such as myAuth.token, in the request body that will contain the token value.
Header name (required): Enter the name of the HTTP header that will contain the token value in the webhook request.
Header scheme (required): Select the authorization scheme used in the header:
-
Bearer expects the header value in the format 'Bearer '
-
Custom lets you define your own scheme prefix
-
None uses the raw header value as the token
Some APIs require you to generate an alternate response message for the initial challenge request.
Challenge detection rules: Set rules to determine when an incoming request should be treated as a challenge. You can build conditions based on the request method, headers, query parameters, or body fields. If the rules evaluate to true, the request is routed through the challenge response path; otherwise, it is handled as a normal request.
Override request media type: The Celigo platform assumes JSON is the media type for success responses for any challenge detected, above. Select an alternative media type if necessary. When you select CSV or XML, you’re presented with additional parsing options in the CSV parser helper or XML parser helper.
Override HTTP response status code: The Celigo platform returns a 200 HTTP status code by default. Enter any alternate successful HTTP status code for successful challenge responses here.
HTTP response headers: Enter the challenge to return in the response header.
HTTP response body: Enter the challenge to return in the response body.
In some cases, you must use a webhook key when generating a cryptographic challenge response. You can complete challenge-response requirements for HMAC signatures using the secrets.key field in the HTTP response body.
The secrets.key field is:
-
Available only in the AFE editor for challenge responses
-
Available only when using HMAC handlebar expressions
-
Available only in the
keyposition of the HMAC expression
Use this section to customize the success response. You can modify headers, status codes, and body content.
HTTP response status code: The Celigo platform returns a 204 HTTP status code for all successful webhook requests by default. Provide an alternative successful HTTP status code for the listener in this setting.
HTTP response headers: Override the HTTP response headers returned for successful requests made to the listener, if necessary.
HTTP response media type: The Celigo platform assumes that JSON is the media type for success responses. Select an alternate media type (XML or plain text), if applicable.
HTTP response body: The Celigo platform returns an empty response body message for all successful webhook requests by default. Customize an HTTP response body returned for successful requests made to the listener, if required.
You can also save mock output data to generic webhook listeners. It works exactly like mock outputs for exports and lookups, except that it is not possible to fetch preview data to populate the mock data field.
If your webhook payload contains records nested inside a wrapper object, you can specify the path to those records so that the export can extract and page them properly.
Override request media type: Choose the expected media type for incoming requests. If left blank (or None is selected), then the webhook auto-detects the media type. This setting is useful for nonstandard Content-Type headers, such as application/cloudevents+json, where the request is properly structuredbut auto-detection would fail.
If you select CSV or XML, you’re presented with additional parsing options in the CSV parser helper or XML parser helper.
Path to records in webhook request body: Enter the path to the array of records inside the webhook payload. The webhook will extract the records at this path and then apply paging based on the page size setting. The maximum payload limit is 50 MB.
This setting accepts JSONPath or normal dot-notation path. Both JSON and XML payloads are supported. For XML payloads, the webhook automatically parses the XML to JSON first, and then the path is evaluated against the resulting JSON. In other words, you are always providing a JSON-based path, even when the original payload is XML.
For example, given the following payload:
{
"orderId": "ORD-1234",
"customer": "Acme Corp",
"lineItems": [
{ "sku": "WIDGET-A", "qty": 10 },
{ "sku": "WIDGET-B", "qty": 5 }
]
}
You would enter lineItems (or $.lineItems) as the path. The export would then produce two records — one for each line item.
Include parent data: Check this box to include data outside the records path as _PARENT on each record. This option is useful when your request body has header-level fields (like order info) and a nested list of line items.
Using the example above with this option enabled, each record would include:
{
"sku": "WIDGET-A",
"qty": 10,
"_PARENT": {
"orderId": "ORD-1234",
"customer": "Acme Corp"
}
}
Open the Advanced section for additional webhook configuration options.
Page size: When an export runs in the context of a data flow (where the data from the export is sent right away to an import queue) the webhook step will break the data being exported into one or more smaller pages of records. Saying this another way, the platform uses streaming to export data out of one app and import it into another app. The Page size setting allows you to specify how many records you want in each page of data. If you leave this option blank, the default page size is 20 records. There is no maximum record limit, but a page of data is limited to 5 MB. Most of the time, the application that you are importing data into will bottleneck the page size value. For example, if you are importing data into NetSuite or Salesforce, they each specify (in their API guides) a maximum number of records that can be submitted in any single request.
Data URI template: When your flow runs but has data errors, this setting can be really helpful in that it allows you to make sure that all the errors in your job dashboard have a link to the original data in the export application. Build a handlebars template to generate the dynamic links based on the data being exported. For example, if you are exporting a customer record from Shopify, you would most likely enter the following value https://your-store.myshopify.com/admin/customers/{{{id}}}. Or, if you are just exporting a CSV file from an FTP site then, the template could simply be one or more columns from the file: {{{internal_id}}, {{{email}}}.
Do not store retry data: Check this box if you do not want to store retry data for records that fail in your flow. Storing retry data can slow down your flow’s overall performance if you are processing very large numbers of records that are failing. Storing retry data also means that anyone with access to your flow’s dashboard can see the retry data in clear text.
Override trace key template: Define a trace key to identify a unique record, overriding the default platform trace key field. You can provide a single field such as {{{field1}}} or handlebars expression. For example, the syntax {{join "_" field1 field2}} will generate a trace key template <field1>_<field2>.