Getting started with the Management API

Getting started with the Management API#

This guide explains some essential information about the Management API.

Management API basics#

The Management API lets you use Adverity through API requests, instead of the Adverity user interface.

Use any HTTP client to interact with the Management API. With Postman, you can use the Adverity Postman collection.

This section of the Adverity documentation contains information about how to make requests to the Management API in order to use the Adverity platform.

Authorizing requests#

Authorize requests to the Management API with a Connect API key, sent in the Authorization header as Bearer {{KEY}}. For more information, see Authorizing to Management API.

API reference#

Every Adverity instance serves the API reference at https://{{INSTANCE}}/api/docs/. Replace {{INSTANCE}} with your Adverity instance.

Code#

This documentation provides examples of requests you can make to the Management API and responses as code blocks. You can copy them and edit them. For example:

curl --location --request GET 'https://{{INSTANCE}}/api/datastreams/' \
--header 'Authorization: Bearer {{KEY}}'

You can import this code as raw text to Postman or to other HTTP clients.

Placeholders#

In the code block above, you may see the red word {{INSTANCE}} capitalized. This marks the text as a placeholder, where you substitute the text with your own value. Postman allows to change placeholders in double curly brackets as variables in an environment. For more information, see Postman documentation.

Placeholders which are not marked with double curly brackets are still capitalized. For example: Substitute KEY_VALUE with the value of your Connect API key. Since these placeholders are not marked with double curly brackets, they are not understood as variables by Postman. If required, substitute them manually.

IDs are marked with placeholder value such as 123.

Difference in naming#

Certain elements require additional explanation or carry different names in the UI compared to the API:

UI Name

API Name

Instance

Your Adverity instance. To find your instance, go to the Adverity user interface. Copy the URL that you see in the browser. Remove https:// from the beginning of the URL and the part similar to /workspace-name/ from the end. For example, if the original URL is https://my-organization.datatap.adverity.com/my-workspace/, the result is my-organization.datatap.adverity.com.

Workspace

Stack

Transformation

Transformer

Destination

Target

Target field

Target column

Issues

Errors

Fetch Jobs

Jobs

Endpoints#

To display the API in your browser, in the following link, replace {{INSTANCE}} with your Adverity instance, and then open the link in your browser: https://{{INSTANCE}}/api/

The base URL for all endpoints:

https://{{INSTANCE}}/api/

Example URL for datastreams endpoint:

https://{{INSTANCE}}/api/datastreams/

Example URL for Jobs endpoint:

https://{{INSTANCE}}/api/jobs/

As mentioned above, with Postman, you can use the Adverity Postman collection to see and test available endpoints and requests.

List of endpoints#

The following list provides information about all available endpoints of the Management API.

Endpoint

Endpoint URL

API request types

Billing Objects

api/billing-objects/

GET

Billings

api/billings/

GET, PATCH

Columns

api/columns/

GET

Connection Invite

api/connection-invite/

POST

Connection Types

api/connection-types/

DELETE, GET, OPTIONS, PATCH, POST

Datastream Types

api/datastream-types/

DELETE, GET, OPTIONS, PATCH, POST

Datastreams

api/datastreams/

GET, OPTIONS, PATCH, POST

Errors

api/errors/

GET

Extracts

api/extracts/

GET

Jobs

api/jobs/

GET, POST

Mapping Tables

api/mapping-tables/

DELETE, GET, PATCH, POST

Notifications

api/notifications/

DELETE, GET, PATCH, POST

Stacks

api/stacks/

DELETE, GET, PATCH, POST

Storage

api/storage/

DELETE, GET, PATCH, POST

Target Columns

api/target-columns/

GET, OPTIONS, POST

Target Types

api/target-types/

DELETE, GET, OPTIONS, POST, PUT

Transformer

api/transformer/

DELETE, GET, PATCH, POST

Value Tables

api/value-tables/

GET

Auth Keys (v1)

api/v1/auth/keys/current/rotate/

POST

Bindings (v1)

api/v1/bindings/

GET, PATCH

Connectors (v1)

api/v1/connectors/

GET

Custom Monitors (v1)

api/v1/custom-monitors/

DELETE, GET, OPTIONS, POST

Datastream Tags (v1)

api/v1/datastream-tags/

DELETE, GET, PATCH, POST

Datastreams (v1)

api/v1/datastreams/

DELETE, GET, OPTIONS, PATCH, POST, PUT

Extracts (v1)

api/v1/extracts/requeue/

POST

Usage (v1)

api/v1/usage/

GET

Users (v1)

api/v1/users/

DELETE, GET, PATCH, POST

Workspaces (v1)

api/v1/workspaces/

GET

Warehouse Query (v1)

api/v1/workspaces/{workspace_id}/warehouse/query/

POST

Warehouse Schema (v1)

api/v1/workspaces/{workspace_id}/warehouse/schema/

GET

API versions#

The Management API has two sets of endpoints:

  • Legacy endpoints (/api/) — the first and broadest API surface. They remain fully supported, and existing integrations keep working as they are. However, their feature set is frozen, meaning no new features will be added here.

  • v1 endpoints (/api/v1/) — the actively developed API surface. This is where we add new features; existing capabilities are brought over and improved over time. v1 endpoints follow their own conventions for keys, identifiers, pagination, and errors. For more information, see Working with v1 endpoints.

v1 does not cover every task yet. Where a task has no v1 endpoint, the legacy endpoint is the right choice. When you build something new, we recommend v1 for the tasks it covers.

The following table shows which endpoints to use for each task:

Task

v1 endpoints

Legacy endpoints

Create and list workspaces

GET /api/v1/workspaces/ (list only)

Creating and listing workspaces in Management API

Configure workspaces

Not available

Configuring workspaces in Management API

Create and configure authorizations

Not available

Creating and configuring authorizations in Management API

Create datastreams

Not available

Creating datastreams in Management API

List datastreams

Listing datastreams in Management API

Configuring datastreams in Management API

Configure datastreams

Enabled state and pulling mode only: Managing pulling mode and smart schedule in Management API

Configuring datastreams in Management API

Tag datastreams

Tagging datastreams in Management API

Not available

Retrieve connector field hierarchies

Retrieving connector field hierarchies in Management API

Not available

Schedule fetches

Smart schedule: Managing pulling mode and smart schedule in Management API

Custom schedules: Scheduling fetches in Management API

Fetch data

Not available

Collecting data in Management API

List jobs

Not available

Managing jobs in Management API

List issues

Not available

Listing issues through Management API

Load historical data to new destinations

Loading historical data to new destinations in Management API

Not available

Configure Data Mapping

Not available

Configuring Data Mapping in Management API

Create and configure transformations

Not available

Creating and configuring transformations in Management API

Configure mapping tables

Not available

Configuring mapping tables in Management API

Query warehouse data

Querying warehouse data in Management API

Not available

Create and delete destinations

Not available

Creating and deleting destinations in Management API

Assign destinations to datastreams

Not available

Assigning destinations to datastreams in Management API

List and update datastream-destination bindings

Managing datastream-destination bindings in Management API

Not available

Configure Data Warehouse and Dashboards binding settings

Not available

Configuring Data Warehouse and Dashboards binding settings in Management API

Manage storage

Not available

Managing storage in Management API

Enable bulk loading for Google BigQuery

Not available

Enabling bulk loading for Google BigQuery in Management API

Assign custom monitors

Bulk assigning custom monitors in Management API

Not available

Manage users

Managing users in Management API

Not available

Configure notifications

Not available

Configuring notifications in Management API

View usage metrics

Viewing workspace usage metrics in Management API

Not available

Refresh and check API keys

Authorizing Management API

Not available

General workflow note#

Often, you can work with the endpoints in the following manner:

  1. Request initial IDs or basic information with a GET request.

  2. Request available configuration options with an OPTIONS request.

  3. Edit the configuration options or create a particular object with a POST or PATCH request.

Additional information#

Rate limits#

Adverity’s Management API has no rate limits. However, each Management API call uses resources, so we recommend that you only perform one Management API task at a time. This avoids overloading your resources and will prevent performance issues. Wait for one Management API call to finish before performing another task.

Timestamps#

Use UTC time zone in all timestamps in the Management API and use the following format:

%Y-%m-%dT%H:%M:%SZ

For example, this translates to the following:

2022-02-01T12:00:22Z

Using filters and ordering data#

All the GET call responses can be filtered by specific parameters. Filter all GET call responses by including the id or date parameters.

To limit the number of results, use the page_size parameter.

https://{{INSTANCE}}/api/jobs/?stack=4&ordering=id&page_size=10

Error responses#

Requests to the legacy endpoints (/api/) that fail with a client error status code, such as 400 or 404, return a response body in the following form:

{
  "detail": "Validation failed",
  "errors": {
    "name": ["This field may not be blank."]
  },
  "name": ["This field may not be blank."]
}

The response body contains the following fields:

  • detail — a summary of the error as a single string. This field is always present.

  • errors — the individual field errors, where the Adverity field name is the key. This field is only present when the request failed validation.

Adverity also keeps the original error fields at the top level of the response body, as shown by the name field in the example above. Use detail and errors in new integrations.

Note

Endpoints that return a list instead of an object in the error response body, such as PUT /api/datastreams/{{DATASTREAM_ID}}/columns/map/, keep returning a list. In this list, the position of each item corresponds to the position of the item you sent in the request.

Server error status codes, such as 500, and the v1 (/api/v1/) endpoints are not affected. For the error responses of the v1 endpoints, see Errors in v1 endpoints.

Working with v1 endpoints#

The v1 endpoints (/api/v1/) work differently from the legacy endpoints (/api/) in a few ways. This section applies to every v1 endpoint unless its guide says otherwise.

Authentication and scopes in v1#

v1 endpoints accept a Connect API key, sent as Authorization: Bearer {{KEY}}. For more information, see Authorizing to Management API.

Each key carries scopes, and each endpoint requires one. The following table shows the scope each area of v1 requires:

Area

Read scope

Write scope

datastreams, datastream tags, connector field hierarchies

datastream:read

datastream:write

Custom monitors

monitor:read

monitor:write

datastream-destination bindings, extract requeueing

destination:read

destination:write

workspaces

workspace:read

Not applicable

Users

user:read

user:write

Warehouse queries and schema

warehouse:read

Not applicable

Row usage

usage:read

Not applicable

A key sees the workspace it was created for and all workspaces below it. Objects in other workspaces are not visible to the key.

Important

A key can only call a v1 endpoint if it has the scope that endpoint needs, for example Datastream: Read-only for listing datastreams. Keys created before scopes existed show only Legacy API in their permissions. They keep working with the legacy endpoints (/api/), but every v1 endpoint rejects them with 403 and a detail such as Missing scopes: datastream:read.

You cannot add permissions to an existing key. To use v1, create a new key with the permissions you need, then switch your integration to it.

To see which permissions a key has, open the Connect API keys page and open the key. For more information, see Authorizing to Management API.

Identifiers in v1#

v1 endpoints identify datastreams, workspaces, tags, and custom monitors by UUID, for example 6f0c2a3e-7d5b-4b8e-9a51-2a8d3f0c9e11, not by the numeric ID that the legacy endpoints use. The uuid field of legacy responses holds the same UUID, so you can map numeric IDs to v1 identifiers. To find workspace UUIDs, use GET /api/v1/workspaces/.

A few v1 endpoints still take a numeric ID where their guide says so, for example the datastream_id filter of GET /api/v1/bindings/.

Pagination in v1#

v1 list endpoints are paginated with the following query parameters:

Parameter

Description

page

The page number, starting at 1. Default: 1.

page_size

The number of items per page. Default: 50. Values above 100 return 100 items.

The response body has the following form:

{
  "items": [],
  "count": 123
}

count is the total number of items across all pages. To read every item, increase page until you have read count items. Unlike the legacy endpoints, the response contains no next or previous links.

Filters in v1#

To pass several values for one filter, repeat the query parameter, for example ?workspace_id={{WORKSPACE_ID_1}}&workspace_id={{WORKSPACE_ID_2}}. Most list filters accept up to 50 values.

Many v1 list endpoints also accept the following workspace filters:

Parameter

Description

workspace_id

Only objects that belong to these workspaces.

workspace_scope

ancestors, descendants, or family: only objects in the workspace given by scope_from_workspace_id and the workspaces above it, below it, or both. The result never includes workspaces the key cannot see.

scope_from_workspace_id

The workspace that workspace_scope starts from. Required when the instance has more than one top-level workspace; otherwise it defaults to the top-level workspace.

The guide of each endpoint lists the filters it supports.

Errors in v1 endpoints#

v1 endpoints return errors with a detail field:

{
  "detail": "Missing scopes: datastream:write."
}

The following status codes are common to all v1 endpoints:

Status code

Meaning

400

The request is valid in form but cannot be carried out, for example because it references an object the key cannot see. detail explains why.

401

The Authorization header is missing, is not a Bearer key, or the key is not valid.

403

The key lacks a required scope.

404

The object does not exist, or the key cannot see it.

409

The request conflicts with the current state of the object, for example an action that does not apply to it right now.

422

The request does not match the expected form, for example a missing field, a value of the wrong type, or a list that is too long. detail contains the list of problems as a JSON-encoded string.

Some endpoints add fields to an error response, such as the existing tag in a 409 response from POST /api/v1/datastream-tags/. Their guides describe these fields.

Contents