# Integrating with Databricks

[Prerequisites](#prerequisites)

[Databricks Guide](#databricks-guide)\
   [Create Personal Access Token](#step-1-create-personal-access-token)\
   [Create and Connect a new SQL Warehouse](#step-2-create-and-connect-a-new-sql-warehouse)\
   [Connect an existing SQL warehouse or all-purpose compute](#step-3-connect-an-existing-sql-warehouse-or-all-purpose-compute)

## Prerequisites

To connect Databricks to Drivetrain, you need:

1. An active Databricks admin account
2. At least one [SQL warehouse](https://docs.databricks.com/en/compute/sql-warehouse/index.html) or a compute cluster to sync data from your catalog.

## Databricks Guide <a href="#databricks-guide" id="databricks-guide"></a>

### Step 1: Create Personal Access Token <a href="#step-1-create-personal-access-token" id="step-1-create-personal-access-token"></a>

1. Drivetrain uses a secure token to connect to Databricks. You can read more about the Databricks token management guide [here](https://docs.databricks.com/en/dev-tools/auth/pat.html)
2. In your Databricks workspace, click your Databricks username in the top bar, and then select **Settings** from the drop down.
3. Click **Developer**.
4. Next to **Access tokens**, click **Manage**.
5. Click **Generate new token**.
6. (Optional) Enter a comment that helps you to identify this token in the future, and change the token’s default lifetime of 90 days. To create a token with no lifetime (not recommended), leave the **Lifetime (days)** box empty (blank).
7. Click **Generate**.
8. Copy the displayed token to a secure location, and then click **Done**.

{% hint style="info" %}
Please note during the data sync, if we find a table which doesn’t have ChangeDataFeed enabled, we try to activate it. Make sure the personal access token has MODIFY permissions on the table. The command to enable ChangeDataFeed for a table is

&#x20;`ALTER TABLE catalog_name.schema_name.table_name SET TBLPROPERTIES (delta.enableChangeDataFeed=true)`
{% endhint %}

### Step 2: Create and Connect a new SQL Warehouse <a href="#step-2-create-and-connect-a-new-sql-warehouse" id="step-2-create-and-connect-a-new-sql-warehouse"></a>

{% hint style="warning" %}
If there is already an unity catalog enabled SQL warehouse or an all-purpose compute present in your Databricks setup, please jump to step 3
{% endhint %}

1. In the Databricks console, go to **SQL** > **SQL warehouses** > **Create SQL warehouse**

<figure><img src="/files/mQbn16Y97zS3NLoRcyyM" alt=""><figcaption></figcaption></figure>

2. In the New SQL warehouse window, enter a **Name** for your warehouse
3. Choose your **Cluster Size** and configure the other warehouse options
4. Click **Create**

<figure><img src="/files/rdqwxhmK01fs6n2gU446" alt=""><figcaption></figcaption></figure>

5. After creating the warehouse, Go to the **Connection details** tab

<figure><img src="/files/5sJ3znNWfcPFrC4kxzVD" alt=""><figcaption></figcaption></figure>

6. Make a note of the following values and keep them in the secured location along with personal access token.
   1. Server Hostname
   2. Port
   3. HTTP Path

### Step 3: Connect an existing SQL warehouse or all-purpose compute <a href="#step-3-connect-an-existing-sql-warehouse-or-all-purpose-compute" id="step-3-connect-an-existing-sql-warehouse-or-all-purpose-compute"></a>

1. Go to your compute cluster or warehouse.
2. Click **Configuration** > **Advanced Options** > **JDBC/ODBC**

<figure><img src="/files/Qd6FbuvkB7zja9O5Zdha" alt=""><figcaption></figcaption></figure>

3. Make a note of the following values and keep them in the secured location along with personal access token.
   1. Server Hostname
   2. Port
   3. HTTP Path

{% hint style="info" %}
That’s it, share server hostname, port, HTTP path & personal access token to your customer success manager.
{% endhint %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://integrations.drivetrain.ai/integration-guide/integrate-with-drivetrain/databases-and-warehouses/integrating-with-databricks.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.