Databricks¶
Connect Adverity Atlas to a Databricks SQL warehouse.
Prerequisites¶
A user or role that can read the data you want to query. Adverity Atlas only issues read queries against your tables, it does not create, modify, or drop them, so read access is all it needs.
What Adverity Atlas reads¶
A sync lists the objects in each schema you bind and records what kind each one is. Profiling then runs aggregate queries over the types it can profile, and those field statistics are what Chat and the glossary work from. An object that is discovered but not profiled is still queryable, it just carries no statistics.
Object type |
Discovered and queryable |
Profiled |
|---|---|---|
Tables: managed, external, streaming, foreign, and shallow clones |
Yes |
Yes |
Views |
Yes |
No |
Materialized views |
Yes |
No |
Querying and profiling need the same access: SELECT on the objects you want available. Profiling
adds no privilege of its own, it reads the same objects with aggregate queries. The principal also
needs USE CATALOG on the Unity Catalog named in Catalog, or no catalogs are listed. The
Connection test on the wizard’s Test step proves that access, in the order a sync needs it:
connect, list the schemas, list the tables in the target schema, and read a row.
Authentication methods¶
Databricks supports 3 authentication methods. Choose the one that matches your security posture.
Access Token (PAT)¶
Use a Databricks personal access token.
Best for: Fastest setup; tied to the issuing user’s permissions.
Credentials:
Field |
Description |
|
|---|---|---|
Access Token |
e.g. |
Required |
OAuth M2M¶
Machine-to-machine OAuth using a Databricks service principal.
Best for: Production service-principal auth with no user dependency.
Note
An admin must first create a Databricks OAuth M2M service config with the service principal’s client ID and secret, under Settings > Integrations > Service Configs. You pick it on the Authenticate step as the OAuth client.
No credentials are stored for this method. The Host field is not shown either: the workspace is
already named on the service config’s Token URL, so Adverity Atlas takes the host from there rather
than asking twice. That is why the Token URL has to be the workspace-level endpoint,
https://<workspace-host>/oidc/v1/token; a client whose Token URL names no workspace is refused
when you save.
The service config is checked as soon as it is saved: Adverity Atlas requests a token from the workspace with the service principal’s credentials and discards it, so a wrong Client Secret or a mistyped Token URL shows as Verification failed on the service config with Databricks’ own reason, before you reach this wizard. The OAuth client pick shows that state next to the client’s name (a verified client shows just its name), and if you pick a failed client anyway, Review & create warns you which one and why. See Service configs.
Azure Service Principal¶
Authenticate via an Azure service principal (client id + secret).
Best for: Azure Databricks workspaces using Entra ID.
Note
An admin must first create a Databricks Azure SP service config with the client credentials, under Settings > Integrations > Service Configs. You pick it on the Authenticate step as the OAuth client.
No credentials are stored for this method. The Host field stays: the service config’s Token URL names your Azure tenant, not a workspace.
Setup¶
Go to Settings > Data Governance > Data Sources. Under Add Connection, choose Databricks. The Add a data source wizard opens and runs in five steps.
Source, confirm Databricks is selected and click Continue.
Auth method, pick one of the methods above and click Continue.
Authenticate, fill in the Configuration fields and either the Credentials fields or, for OAuth M2M and Azure Service Principal, the OAuth client pick, then click Save credentials and Continue.
Test, Adverity Atlas tests the connection as soon as you arrive and shows the result under Connection test. Continue unlocks once the test passes. After fixing a failure, click Test again. Back discards the saved credentials and returns you to Authenticate, with the configuration still filled in.
Review & create, enter a Name, choose a Sync schedule, and pick a Mode: Active (sync runs on schedule and agents can query this source), Development (sync runs, but agents cannot query it yet), or Disabled (no automated operations). Then click Create data source.
The Connection test runs the full connection diagnostic: it authenticates, lists the visible schemas, lists the tables in the target schema, and reads a row. You cannot continue to Review & create until every check passes, so a wrong credential or a missing grant is resolved here rather than becoming an unhealthy source afterwards. A failed check reports the warehouse’s own error, and the grant that is missing where Adverity Atlas can work it out, see Connection diagnostics. A test that could not run at all says “The connection test could not run. Try again in a moment.”
Adverity Atlas verifies the connection as soon as the source is created, and starts the first sync when verification succeeds and the mode allows it. Schema discovery and field profiling run from that sync.
Tip
If someone else holds the credentials, use Share a one-time link under Or ask someone else on the Authenticate step. They submit the credentials through that link and see the connection test run on their page too. If it fails, Provide credentials again lets them correct what they entered. Your wizard moves on to Test once they submit, so the secret never passes through you. The link is not offered for OAuth M2M and Azure Service Principal, where the service config already holds the credentials.
Connection fields¶
These appear under Configuration on the wizard’s Authenticate step.
Field |
Description |
|
|---|---|---|
Host |
The workspace hostname, e.g. |
Required, except for OAuth M2M, where it is taken from the OAuth client’s Token URL and not shown |
HTTP Path |
The SQL warehouse’s HTTP path, e.g. |
Required |
Catalog |
The Unity Catalog whose schemas Adverity Atlas syncs, e.g. |
Required |
Adverity Atlas checks these as you save them and shows what to fix under the field: a full URL
pasted into Host, or a cluster path in HTTP Path (Adverity Atlas connects to SQL warehouses,
not to all-purpose clusters). Connections always use port 443.
Note
Catalog has to be filled in. Every introspection query names a catalog, so a blank one connects successfully and then fails at List tables in the connection test.
Relationships¶
On a Unity Catalog catalog, Adverity Atlas reads the foreign-key constraints declared on your tables and lists them under each asset’s Relationships as Confirmed joins, including composite keys and keys that reference a table in another schema of the same catalog. A key that references a table in a different catalog is not listed.
Databricks does not enforce these constraints; it records them as informational. Adverity Atlas trusts the declaration as written, the same way it does for Snowflake, so a Confirmed join says the constraint exists, not that the warehouse guarantees the rows match.
The legacy Hive metastore has no constraints. Relationships on those data sources come from inference alone.
Sync freshness¶
After the first sync, Adverity Atlas re-reads a table’s columns only when the table has changed, and
re-profiles its data only when it has changed or the profile is more than a week old. For Databricks
the change signal is the time of the table’s last Delta commit, read with DESCRIBE DETAIL, one
statement per table. It moves on inserts, updates, deletes and schema changes, and not on reads.
Only Delta tables carry that signal. Views, foreign tables and tables in other formats have none, so
they are re-read and re-profiled on every sync, as before. The principal needs SELECT on a table
to describe it; a table it cannot describe is treated the same way.
Troubleshooting¶
Symptom |
Likely cause |
|---|---|
“Invalid access token” |
The PAT expired or was revoked. Issue a new one. |
“Warehouse is stopped” |
Start the SQL warehouse or enable auto-start. |
Service config shows “Verification failed” |
The reason under the badge is Databricks’ own. “Invalid client credentials” means the service principal’s Client ID or Client Secret; “Could not reach the authorization server” means the Token URL. Fix and save to check again. |
No catalogs listed |
The principal lacks USE CATALOG on Unity Catalog. |
“The token URL of the OAuth client … does not name a workspace” |
The service config’s Token URL is the account-level or the Entra endpoint. Change it to
|
What’s next?¶
Configure the glossary, align the language across your platforms so answers use your definitions
Ask questions about your data, start a conversation against the data you just connected
Data sources, schema discovery, profiling, and managing connections