Set up a connection to Databricks

Databricks is a unified analytics platform that provides a collaborative environment for data science, data engineering, and machine learning.

Note: Celigo uses a JDBC connection to connect to Databricks; we use the databricks-jdbc-2.6.33.jar. If you would like to connect to any application that supports this JAR file but is currently not available in our connector list, you're welcome to submit it as an idea on the Product portal.

Set up a connection

After you start the connection, configure it in the Create connection panel and complete all of the required * authentication settings:

Setting	Instructions
Name *	Enter a clear and distinguishable name for your connection. Throughout integrator.io imports and exports, you will have the option to choose this new connection. A unique identifier will prove helpful later when selecting it from a list of the connections in your account.
Host *	Enter the connection string/URL to the server that hosts the database. For example, <protocol>://<host>:<port>/[?<properties>] jdbc://localhost:5001/[?properties] How to find host details Sign in to Databricks. In the Account console, click Workspaces. Select a workspace and click Open at the end of the row. Then, enter your credentials to authenticate. In the navigation pane of the workspace, select Compute. Then click the required cluster. In the cluster page (default Configuration tab) under Advanced options, select JDBC/ODBC. In the JDBC URL, view and copy details from Databricks. This JDBC URL provides the host value. (Paste this string in the Host field in the Celigo platform. Delete the UID and PWD parts from the JDBC connection string when you paste the value. Also, make sure there's no trailing semi-colon.) Note: Do not include your username or password as a part of the host value (JDBC connection string) because it is stored as plain text. Instead, you can provide it in the fields below. Tip: Your host URL should start with jdbc:databricks:. This is automatically prepended to the value you enter in the Host field if it's not included.
Username	Enter your user name for authentication to Databricks.
Password	Enter your password for authentication to Databricks.
Configure properties	Define additional configurations for the database connection. For example, you can configure the connection timeout if the server you are trying to connect to is slow.
Borrow concurrency from	Select another connection from the list. By default, all data flowing through a connection record is submitted to the respective endpoint application at the Concurrency level configured for that connection record. There are cases, however, where multiple integrator.io connections need to share the same concurrency level, and this field allows you to specify which connection you would like to borrow from. With borrowed concurrency, the data flowing through both connections will be submitted to the endpoint application together, via a shared concurrency model.
Concurrency level	Set this field to limit the number of concurrent HTTP requests (up to 25) allowed by the connection resource at any one time, or leave this field blank to use burst mode. With burst mode, integrator.io will make HTTP requests as fast as possible, with high levels of concurrency.
Configure encrypted fields	Store all sensitive fields required by your imports and exports to access the app you are connecting to. Click the Expand window (  ) button to modify the JSON sent to the app in the Encrypted Advanced Field Editor. For example, enter {"secretKey": "someSecretKeyValue"}. (You can also retrieve these sensitive values elsewhere, such as How to test this connection?, via the example handlebars format {{{connection.http.encrypted.secretKey}}}.) Multiple layers of protection are in place, including AES 256 encryption, to keep your connection’s encrypted fields safe. When editing this form later, you must enter this value again; it is stored only when the connection is saved and never displayed as text.

After configuring the settings, click Test Connection. When the connection is verified, you will receive the following message:

The connection is now added to your account. 

• You can see it listed under Resources > Connections.
• If a connection is part of an integration, you can see it listed under Home > Integration name > Connections, and you may proceed to register the connection.