Drivetrain Integration Guide
  • INTEGRATION GUIDE
    • Integrate with Drivetrain
      • Accounting & ERP
        • Integrating with Certinia
        • Integrating with NetSuite
        • Integrating with QuickBooks
        • Integrating with Rillet
        • Integrating with Sage Intacct
        • Integrating with Tally
        • Integrating with Xero
        • Integrating with Zoho Books
      • BI
        • Integrating with Looker
        • Integrating with Tableau
      • Billing & Invoicing
        • Integrating with Chargebee
        • Integrating with Invoiced
        • Integrating with Maxio Chargify
        • Integrating with Maxio SaaSOptics
        • Integrating with Recurly
        • Integrating with Stripe
        • Integrating with Subscript
      • CRM
        • Integrating with Active Campaign
        • Integrating with HubSpot
        • Integrating with LeadSquared
        • Integrating with Outreach
        • Integrating with Pipedrive
        • Integrating with Salesforce
      • Customer Success
        • Integrating with ChurnZero
        • Integrating with Freshdesk
        • Integrating with Planhat
        • Integrating with Zendesk
      • Databases and Warehouses
        • Integrating with Azure SQL Server
        • Integrating with BigQuery
        • Integrating with Databricks
        • Integrating with Redshift
        • Integrating with S3
        • Integrating with Snowflake
        • Integrating with Snowflake (Small Datasets)
        • Integrating with Google Cloud Storage
        • Integrating with Postgres
      • HRIS & ATS
        • Integrating with ADP
        • Integrating with ADP Workforce Now (SFTP)
        • Integrating with Ashby
        • Integrating with BambooHR
        • Integrating with Darwinbox
        • Integrating with Deel
        • Integrating with Greenhouse
        • Integrating with Gusto
        • Integrating with HiBob
        • Integrating with Justworks
        • Integrating with Lever
        • Integrating with Namely
        • Integrating with Paylocity
        • Integrating with Personio
        • Integrating with Sapling
        • Integrating with UKG Pro
        • Integrating with Zenefits
      • Spreadsheets
        • Integrating with Airtable
        • Integrating with Google Drive
        • Integrating with Google Sheets
        • Integrating with Microsoft OneDrive
      • Others
        • Integrating with Brex
        • Integrating with Clari
        • Integrating with NetSuite (Role Based Access)
        • Integrating with Shopify
        • Integrating with Twilio
        • Integrating with Officevibe
        • Integrating with SFTP
        • Integrating with Drivetrain's SFTP
        • Setting up SSO
Powered by GitBook
On this page
  • Prerequisites
  • Databricks Guide
  • Step 1: Create Personal Access Token
  • Step 2: Create and Connect a new SQL Warehouse
  • Step 3: Connect an existing SQL warehouse or all-purpose compute

Was this helpful?

  1. INTEGRATION GUIDE
  2. Integrate with Drivetrain
  3. Databases and Warehouses

Integrating with Databricks

PreviousIntegrating with BigQueryNextIntegrating with Redshift

Last updated 2 months ago

Was this helpful?

Prerequisites

To connect Databricks to Drivetrain, you need:

  1. An active Databricks admin account

  2. At least one or a compute cluster to sync data from your catalog.

Databricks Guide

Step 1: Create Personal Access Token

  1. Drivetrain uses a secure token to connect to Databricks. You can read more about the Databricks token management guide

  2. In your Databricks workspace, click your Databricks username in the top bar, and then select Settings from the drop down.

  3. Click Developer.

  4. Next to Access tokens, click Manage.

  5. Click Generate new token.

  6. (Optional) Enter a comment that helps you to identify this token in the future, and change the token’s default lifetime of 90 days. To create a token with no lifetime (not recommended), leave the Lifetime (days) box empty (blank).

  7. Click Generate.

  8. Copy the displayed token to a secure location, and then click Done.

Please note during the data sync, if we find a table which doesn’t have ChangeDataFeed enabled, we try to activate it. Make sure the personal access token has MODIFY permissions on the table. The command to enable ChangeDataFeed for a table is

ALTER TABLE catalog_name.schema_name.table_name SET TBLPROPERTIES (delta.enableChangeDataFeed=true)

Step 2: Create and Connect a new SQL Warehouse

If there is already an unity catalog enabled SQL warehouse or an all-purpose compute present in your Databricks setup, please jump to step 3

  1. In the Databricks console, go to SQL > SQL warehouses > Create SQL warehouse

  1. In the New SQL warehouse window, enter a Name for your warehouse

  2. Choose your Cluster Size and configure the other warehouse options

  3. Click Create

  1. After creating the warehouse, Go to the Connection details tab

  1. Make a note of the following values and keep them in the secured location along with personal access token.

    1. Server Hostname

    2. Port

    3. HTTP Path

Step 3: Connect an existing SQL warehouse or all-purpose compute

  1. Go to your compute cluster or warehouse.

  2. Click Configuration > Advanced Options > JDBC/ODBC

  1. Make a note of the following values and keep them in the secured location along with personal access token.

    1. Server Hostname

    2. Port

    3. HTTP Path

That’s it, share server hostname, port, HTTP path & personal access token to your customer success manager.

SQL warehouse
here
Prerequisites
Databricks Guide
Create Personal Access Token
Create and Connect a new SQL Warehouse
Connect an existing SQL warehouse or all-purpose compute