Set up a connection to Confluent Kafka

Rachel Shanthi Bijja

Updated November 22, 2024 11:04

Confluent Kafka is a real-time data streaming technology capable of handling trillions of events per day.

Set up a connection

This video demonstrates creating a Confluent Kafka connection.

After you start the connection , configure it in the Create connection panel and complete all of the required * authentication settings:

Setting	Instructions
Name your connection *	Enter a clear and distinguishable name. Throughout integrator.io imports and exports, you will have the option to choose this new connection. A unique identifier will prove helpful later when selecting it from a list of the connections in your account.
REST endpoint *	Enter your Confluent Kafka REST endpoint. $Fetch data {//theme.zdassets.com/theme_assets/1098969/5157503509223f89f9b3e7578d2ca78ce882fc18.svg}$ How to retrieve REST endpoint
API key *	Enter your Confluent Kafka account API key. $Fetch data {//theme.zdassets.com/theme_assets/1098969/5157503509223f89f9b3e7578d2ca78ce882fc18.svg}$ How to retrieve API key
API secret *	Enter your Confluent Kafka API secret. Multiple layers of protection are in place, including AES 256 encryption, to keep your connection’s API secret safe. When editing this form later, you must enter this value again; it is stored only for a saved connection. $Fetch data {//theme.zdassets.com/theme_assets/1098969/5157503509223f89f9b3e7578d2ca78ce882fc18.svg}$ How to retrieve API secret
Cluster ID *	Enter your Confluent Kafka account cluster ID. $Fetch data {//theme.zdassets.com/theme_assets/1098969/5157503509223f89f9b3e7578d2ca78ce882fc18.svg}$ How to retrieve cluster ID

How to retrieve REST endpoint

Sign in to your Confluent Kafka account.
Navigate to Settings > Admistrator > Environments and select the environment.
Select the required Cluster. The Cluster overview page opens. For more information on how to create a cluster, see Create clusters.
Navigate to Cluster settings > General > Endpoints.
Copy REST endpoint.

How to retrieve API key and API secret

Sign in to your Confluent Kafka account.
Navigate to Settings > Admistrator > Environments and select the environment.
Select the required Cluster. The Cluster overview page opens.
Navigate to API keys.
Click +Add key, select the required access, and click Next.
Copy API key and Secret.

Note

Click Download and continue to save the key and secret. If you do not download your new secret, you cannot see it again.

How to retrieve the Cluster ID

Sign in to your Confluent Kafka account.
Navigate to Settings > Admistrator > Environments and select the environment.
Select the required Cluster. The Cluster overview page opens. For more information on how to create a cluster, see Create clusters, see Create clusters.
Navigate to Cluster settings > General > Identification.
Copy Cluster ID.

Tip

This connector documentation describes only the settings shown for the Simple view. For the corresponding HTTP settings, see basic universal connector documentation.