Synchronize with Databricks

Link ​ValidMind inventory records to models, datasets, and agents in Databricks Unity Catalog for governance across your ML ecosystem and bidirectional metadata synchronization.

You can also run validation notebooks directly against Databricks-hosted data.

How does the integration with Databricks work?

Databricks Unity Catalog provides a unified governance solution for all data and AI assets across your Databricks workspaces. The integration exposes three Unity Catalog resource types:

• Models — Registered ML models in Unity Catalog’s model registry.
• Datasets — Tables and datasets managed by Unity Catalog.
• Agents — AI agents and applications built on Databricks.

After linking your inventory record to Databricks, you can run validation tests directly against data hosted in Unity Catalog.

flowchart LR
    subgraph databricks [Databricks]
        UC[Unity Catalog]
        Data[Tables/Datasets]
    end
    
    subgraph notebook [Validation Notebook]
        SDK[ValidMind SDK]
    end
    
    subgraph validmind [ValidMind Platform]
        Report[Validation Report]
        Inventory[Inventory Record]
    end
    
    UC --> Data
    Data --> SDK
    SDK --> Report
    Report --> Inventory
    Inventory -.->|linked| UC

Data flow between Databricks and ValidMind

Prerequisites

• You are logged in to ValidMind.
• You are a Customer Admin or assigned another role with sufficient permissions to configure connections.¹
• A secret is configured for your Databricks Personal Access Token.²
• You have admin access to your Databricks workspace.

¹ Manage permissions

² Manage secrets

Sync ​ValidMind records with Databricks

1. Configure Databricks connection

1. In the left sidebar, click Settings.

2. Under Integrations, select Connections.

3. Click Add Connection.

4. In the modal that opens, select Databricks.³

5. Complete:

   • Integration Name — How other admins can identify the connection.
   • Description (optional) — The intended usage or additional details.
   • Workspace URL — Your Databricks workspace URL, found in the browser address bar, such as https://adb-1234567890.azuredatabricks.net.
   • SQL Warehouse ID — The ID of your SQL Warehouse, found in SQL Warehouses settings.
   • Personal Access Token — Select a secret containing your Databricks Personal Access Token.
   • Initial Status — Set to Operational to enable immediately or Disabled if you plan to finish setup later.

6. Click Save Integration.

7. Test the connection:

   1. Hover over the Databricks connection you just created.
   2. When the menu appears, click on it and select Test Connection.

   If the test is successful, the message Connection successful is displayed.

³ 

Configure the Databricks connection

2. Link records to Databricks

Once the connection is configured, you can link ​ValidMind inventory records to your Databricks Unity Catalog resources:

1. In the left sidebar, click Inventory.

2. Select a record by clicking on it or find your record by applying a filter or searching for it.⁴

3. Scroll down until you locate the Databricks connection box in the right sidebar.

4. Hover over the Databricks box.

5. When the menu appears, click on it and select Link Model.

6. In the modal that opens:

   • Select Model — Choose the Databricks model to link from the dropdown.
   • Sync Frequency — Set how often ValidMind automatically syncs data from the external system.

7. Click Link Model to complete the link.

⁴ Working with the inventory

After linking, metadata from the Unity Catalog resource syncs to ​ValidMind. You can use linked fields in custom calculated fields to surface Databricks metadata directly in your inventory views.

What’s next