Data sources API¶

Connect warehouses, sync and browse their schema, profile fields, and resolve duplicate data. See Data sources.

Paths are relative to https://<your-atlas-host>/api/v1. For the API key and headers every request needs, see Authentication.

Endpoint	What it does
POST /datasources/	Create a new data source connection
GET /datasources/	List data sources, optionally filtered by workspace
GET /datasources/search	Cross-datasource metadata search
GET /datasources/{datasource_id}	Get a data source by ID
PATCH /datasources/{datasource_id}	Update a data source
DELETE /datasources/{datasource_id}	Delete a data source
POST /datasources/{datasource_id}/verify	Test the connection for a data source
POST /datasources/{datasource_id}/sync	Start a background metadata sync for a data source
POST /datasources/{datasource_id}/diagnose	Start a background connection diagnostic; returns the task execution id to poll
GET /datasources/{datasource_id}/collections	List collections for a data source
GET /datasources/collections/{collection_id}/assets	List assets in a collection
GET /datasources/assets/{asset_id}/fields	List fields for an asset
GET /datasources/assets/{asset_id}/relationships	List relationships for an asset
POST /datasources/assets/{asset_id}/profile	Start background field profiling for a data asset
POST /datasources/{datasource_id}/profile-bulk	Start background field profiling for several data assets
GET /datasources/assets/{asset_id}/profiles	Get the latest profile for each field in an asset
GET /datasources/fields/{field_id}/profile	Get the latest profile for a single field
GET /datasources/overlaps	List detected data overlaps (duplicate data across datasources)
POST /datasources/overlaps/{overlap_id}/resolve	Resolve an overlap by hiding one of the duplicate assets in the workspace
POST /datasources/overlaps/{overlap_id}/dismiss	Dismiss an overlap (will not resurface for this pair)

Create data source¶

POST /datasources/

Create a new data source connection.

Request body

Field	Type	Required	Description
`name`	string	Yes	Unique within your organization.
`source_type`	string	Yes	Warehouse type: `postgres`, `snowflake`, `bigquery`, `databricks`, `athena`, `redshift` or `trino`.
`category`	string		Ignored; derived from `source_type`.
`initial_workspace_id`	UUID		Workspace the source is first shared with; it holds the staged data. Needs admin rights on it.
`workspace_ids`	list of UUID		More workspaces to share the source with. Needs admin rights on each.
`config`	object		Connection settings; the keys depend on `source_type`, such as `project_id` for BigQuery or `host` and `database` for Postgres. `auth_method` picks the credentials shape.
`credentials`	object		Secret login details; which keys are required depends on `source_type` and `config.auth_method`, such as `service_account_json` for BigQuery.
`include_patterns`	list of string		Only sync schemas (BigQuery datasets) whose name matches one of these case-sensitive glob patterns, such as `marketing_*`. Empty means all.
`exclude_patterns`	list of string		Skip schemas whose name matches one of these glob patterns. Wins over `include_patterns`.
`service_config_id`	UUID		An OAuth client configuration; required for Databricks `oauth_m2m` and `azure_sp`.
`schedule`	string		Cron expression, in UTC, for automatic metadata syncs. Empty removes the schedule.
`mode`	string		`dev`: syncs run but the assistant cannot query the source. `active`: both allowed. `disabled`: everything paused.

Response

Field	Type
`id`	UUID
`name`	string
`source_type`	string
`category`	string
`config`	object
`has_credentials`	boolean
`include_patterns`	list of string
`exclude_patterns`	list of string
`is_verified`	boolean
`verified_at`	datetime
`last_error`	string
`health_status`	string
`last_health_check_at`	datetime
`mode`	string
`last_sync_execution_id`	UUID
`last_sync_status`	string
`last_sync_completed_at`	datetime
`service_config_id`	UUID
`schedule`	string
`schedule_active`	boolean
`last_scheduled_at`	datetime
`created_at`	datetime
`updated_at`	datetime

List data sources¶

GET /datasources/

List data sources, optionally filtered by workspace.

Parameters

Name	In	Type	Required
`workspace_id`	query	UUID	Yes

Response A list of objects:

Field	Type
`id`	UUID
`name`	string
`source_type`	string
`category`	string
`config`	object
`has_credentials`	boolean
`include_patterns`	list of string
`exclude_patterns`	list of string
`is_verified`	boolean
`verified_at`	datetime
`last_error`	string
`health_status`	string
`last_health_check_at`	datetime
`mode`	string
`last_sync_execution_id`	UUID
`last_sync_status`	string
`last_sync_completed_at`	datetime
`service_config_id`	UUID
`schedule`	string
`schedule_active`	boolean
`last_scheduled_at`	datetime
`created_at`	datetime
`updated_at`	datetime

Search data source metadata¶

GET /datasources/search

Cross-datasource metadata search.

Parameters

Name	In	Type	Required
`q`	query	string	Yes
`scope`	query	string
`limit`	query	integer

Response A list of objects:

Field	Type
`id`	UUID
`name`	string
`result_type`	string
`path`	string
`datasource_id`	UUID
`asset_id`	UUID
`datasource_name`	string
`collection_name`	string
`asset_name`	string
`field_name`	string
`comment`	string
`data_type`	string

Get data source¶

GET /datasources/{datasource_id}

Get a data source by ID.

Parameters

Name	In	Type	Required
`datasource_id`	path	UUID	Yes

Response

Field	Type
`id`	UUID
`name`	string
`source_type`	string
`category`	string
`config`	object
`has_credentials`	boolean
`include_patterns`	list of string
`exclude_patterns`	list of string
`is_verified`	boolean
`verified_at`	datetime
`last_error`	string
`health_status`	string
`last_health_check_at`	datetime
`mode`	string
`last_sync_execution_id`	UUID
`last_sync_status`	string
`last_sync_completed_at`	datetime
`service_config_id`	UUID
`schedule`	string
`schedule_active`	boolean
`last_scheduled_at`	datetime
`created_at`	datetime
`updated_at`	datetime

Update data source¶

PATCH /datasources/{datasource_id}

Update a data source.

Parameters

Name	In	Type	Required
`datasource_id`	path	UUID	Yes

Request body

Field	Type	Description
`name`	string
`config`	object	Connection settings; the keys depend on `source_type`, such as `project_id` for BigQuery or `host` and `database` for Postgres. `auth_method` picks the credentials shape.
`credentials`	object	Secret login details; which keys are required depends on `source_type` and `config.auth_method`, such as `service_account_json` for BigQuery. Changing them resets the verification status.
`include_patterns`	list of string	Only sync schemas (BigQuery datasets) whose name matches one of these case-sensitive glob patterns, such as `marketing_*`. Empty means all.
`exclude_patterns`	list of string	Skip schemas whose name matches one of these glob patterns. Wins over `include_patterns`.
`service_config_id`	UUID	An OAuth client configuration; required for Databricks `oauth_m2m` and `azure_sp`. Send `null` to remove the link.
`schedule`	string	Cron expression, in UTC, for automatic metadata syncs. Empty removes the schedule.
`schedule_active`	boolean	Pause (`false`) or resume (`true`) the existing sync schedule.
`mode`	string	`dev`: syncs run but the assistant cannot query the source. `active`: both allowed. `disabled`: everything paused.

Response

Field	Type
`id`	UUID
`name`	string
`source_type`	string
`category`	string
`config`	object
`has_credentials`	boolean
`include_patterns`	list of string
`exclude_patterns`	list of string
`is_verified`	boolean
`verified_at`	datetime
`last_error`	string
`health_status`	string
`last_health_check_at`	datetime
`mode`	string
`last_sync_execution_id`	UUID
`last_sync_status`	string
`last_sync_completed_at`	datetime
`service_config_id`	UUID
`schedule`	string
`schedule_active`	boolean
`last_scheduled_at`	datetime
`created_at`	datetime
`updated_at`	datetime

Delete data source¶

DELETE /datasources/{datasource_id}

Delete a data source.

Parameters

Name	In	Type	Required
`datasource_id`	path	UUID	Yes

Response

The raw file content.

Verify data source¶

POST /datasources/{datasource_id}/verify

Test the connection for a data source.

Parameters

Name	In	Type	Required
`datasource_id`	path	UUID	Yes

Response

Field	Type
`id`	UUID
`name`	string
`source_type`	string
`category`	string
`config`	object
`has_credentials`	boolean
`include_patterns`	list of string
`exclude_patterns`	list of string
`is_verified`	boolean
`verified_at`	datetime
`last_error`	string
`health_status`	string
`last_health_check_at`	datetime
`mode`	string
`last_sync_execution_id`	UUID
`last_sync_status`	string
`last_sync_completed_at`	datetime
`service_config_id`	UUID
`schedule`	string
`schedule_active`	boolean
`last_scheduled_at`	datetime
`created_at`	datetime
`updated_at`	datetime

Sync data source¶

POST /datasources/{datasource_id}/sync

Start a background metadata sync for a data source.

Parameters

Name	In	Type	Required
`datasource_id`	path	UUID	Yes

Response

Field	Type
`id`	UUID
`name`	string
`source_type`	string
`category`	string
`config`	object
`has_credentials`	boolean
`include_patterns`	list of string
`exclude_patterns`	list of string
`is_verified`	boolean
`verified_at`	datetime
`last_error`	string
`health_status`	string
`last_health_check_at`	datetime
`mode`	string
`last_sync_execution_id`	UUID
`last_sync_status`	string
`last_sync_completed_at`	datetime
`service_config_id`	UUID
`schedule`	string
`schedule_active`	boolean
`last_scheduled_at`	datetime
`created_at`	datetime
`updated_at`	datetime

Diagnose data source¶

POST /datasources/{datasource_id}/diagnose

Start a background connection diagnostic; returns the task execution id to poll.

Parameters

Name	In	Type	Required
`datasource_id`	path	UUID	Yes

Response

The raw file content.

List collections¶

GET /datasources/{datasource_id}/collections

List collections for a data source.

Parameters

Name	In	Type	Required
`datasource_id`	path	UUID	Yes

Response A list of objects:

Field	Type
`id`	UUID
`name`	string
`comment`	string
`location`	string
`item_count`	integer
`created_at`	datetime
`updated_at`	datetime

List assets¶

GET /datasources/collections/{collection_id}/assets

List assets in a collection.

Parameters

Name	In	Type	Required
`collection_id`	path	UUID	Yes
`exclude_views`	query	boolean

Response A list of objects:

Field	Type
`id`	UUID
`name`	string
`comment`	string
`asset_type`	string
`row_count`	integer
`field_count`	integer
`created_at`	datetime
`updated_at`	datetime

List fields¶

GET /datasources/assets/{asset_id}/fields

List fields for an asset.

Parameters

Name	In	Type	Required
`asset_id`	path	UUID	Yes

Response A list of objects:

Field	Type
`id`	UUID
`name`	string
`data_type`	string
`is_nullable`	boolean
`is_primary_key`	boolean
`default_value`	string
`comment`	string
`ordinal_position`	integer
`null_rate`	number
`distinct_count`	integer
`has_pii`	boolean
`pii_types`	list of string

List relationships¶

GET /datasources/assets/{asset_id}/relationships

List relationships for an asset.

Parameters

Name	In	Type	Required
`asset_id`	path	UUID	Yes

Response A list of objects:

Field	Type
`id`	UUID
`name`	string
`source_asset_id`	UUID
`source_asset_name`	string
`source_fields`	list of string
`target_asset_id`	UUID
`target_asset_name`	string
`target_fields`	list of string
`relationship_type`	string
`cardinality`	string
`is_confirmed`	boolean
`confidence`	number
`evidence`	object

Profile asset¶

POST /datasources/assets/{asset_id}/profile

Start background field profiling for a data asset.

Parameters

Name	In	Type	Required
`asset_id`	path	UUID	Yes

Response

Field	Type
`id`	UUID
`name`	string
`source_type`	string
`category`	string
`config`	object
`has_credentials`	boolean
`include_patterns`	list of string
`exclude_patterns`	list of string
`is_verified`	boolean
`verified_at`	datetime
`last_error`	string
`health_status`	string
`last_health_check_at`	datetime
`mode`	string
`last_sync_execution_id`	UUID
`last_sync_status`	string
`last_sync_completed_at`	datetime
`service_config_id`	UUID
`schedule`	string
`schedule_active`	boolean
`last_scheduled_at`	datetime
`created_at`	datetime
`updated_at`	datetime

Profile several assets¶

POST /datasources/{datasource_id}/profile-bulk

Start background field profiling for several data assets.

Parameters

Name	In	Type	Required
`datasource_id`	path	UUID	Yes

Request body

Field	Type	Required
`datasource_id`	UUID	Yes
`asset_ids`	list of UUID	Yes

Response

The raw file content.

List profiles¶

GET /datasources/assets/{asset_id}/profiles

Get the latest profile for each field in an asset.

Parameters

Name	In	Type	Required
`asset_id`	path	UUID	Yes

Response A list of objects:

Field	Type
`id`	UUID
`field_id`	UUID
`field_name`	string
`null_count`	integer
`total_count`	integer
`null_rate`	number
`distinct_count`	integer
`distinct_rate`	number
`min_value`	string
`max_value`	string
`mean_value`	number
`sample_values`	list of string
`created_at`	datetime
`median_value`	number
`stddev_value`	number
`percentiles`	object
`top_values`	list of object
`histogram`	list of object
`caveats`	list of object

Get field profile¶

GET /datasources/fields/{field_id}/profile

Get the latest profile for a single field.

Parameters

Name	In	Type	Required
`field_id`	path	UUID	Yes

List overlaps¶

GET /datasources/overlaps

List detected data overlaps (duplicate data across datasources).

Parameters

Name	In	Type
`status`	query	string
`workspace_id`	query	UUID

Response A list of objects:

Field	Type
`id`	UUID
`source_asset_id`	UUID
`source_asset_name`	string
`source_datasource_name`	string
`target_asset_id`	UUID
`target_asset_name`	string
`target_datasource_name`	string
`overlap_type`	string
`confidence`	number
`evidence`	object
`status`	string
`resolved_by_email`	string
`resolved_binding_id`	UUID
`hidden_asset_id`	UUID
`created_at`	datetime

Resolve overlap¶

POST /datasources/overlaps/{overlap_id}/resolve

Resolve an overlap by hiding one of the duplicate assets in the workspace.

Parameters

Name	In	Type	Required
`overlap_id`	path	UUID	Yes

Request body

Field	Type	Required	Description
`hide_asset_id`	UUID	Yes	The duplicate table to hide in the workspace; one of the overlap’s two tables.
`reason`	string		Ignored; kept for compatibility.

Response

Field	Type
`id`	UUID
`source_asset_id`	UUID
`source_asset_name`	string
`source_datasource_name`	string
`target_asset_id`	UUID
`target_asset_name`	string
`target_datasource_name`	string
`overlap_type`	string
`confidence`	number
`evidence`	object
`status`	string
`resolved_by_email`	string
`resolved_binding_id`	UUID
`hidden_asset_id`	UUID
`created_at`	datetime

Dismiss overlap¶

POST /datasources/overlaps/{overlap_id}/dismiss

Dismiss an overlap (will not resurface for this pair).

Parameters

Name	In	Type	Required
`overlap_id`	path	UUID	Yes

Response

Field	Type
`id`	UUID
`source_asset_id`	UUID
`source_asset_name`	string
`source_datasource_name`	string
`target_asset_id`	UUID
`target_asset_name`	string
`target_datasource_name`	string
`overlap_type`	string
`confidence`	number
`evidence`	object
`status`	string
`resolved_by_email`	string
`resolved_binding_id`	UUID
`hidden_asset_id`	UUID
`created_at`	datetime

What’s next?¶

API overview: authentication, worked examples, and error handling
Webhooks: get events pushed to your systems instead of polling
Service accounts: create the key an integration uses