Creating datastreams in Management API

Creating datastreams in Management API#

This article explains how to create datastreams with the Management API.

Prerequisites#

Before you complete the procedure in this guide, perform all of the following actions:

Determine the ID of the workspace where you want to create the datastream. For more information, see Listing existing workspaces.
Create an authorization for the datastream type and determine the authorization ID. For more information, see Creating and configuring authorizations in Management API.

Determining a datastream type’s ID#

To determine a datastream type’s ID, follow these steps:

Note

You can also use this request to get information about the data source API in the api_details parameter of the response.

Create a GET request to the following endpoint:
```
https://{{INSTANCE}}/api/datastream-types/
```
In the HTTP request header, include the parameter Authorization with one of the following values:
- Token {{KEY}} if you use a key generated with user credentials in Management API.
- Bearer {{KEY}} if you use a key generated in the Adverity user interface.
Send the request.

As a result, you obtain the available datastream types and their IDs in the response.

Note

If you already know the datastream type’s API name, create a GET request to the following endpoint in step 1.

https://{{INSTANCE}}/api/datastream-types/?search={{DATASTREAM_TYPE_NAME}}

You can also use this method to search for part of the data source name. For example, using ?search=google returns all datastream types whose name contains the word google.

If you use this method, the JSON response only includes the datastream type you specify.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request GET 'https://{{INSTANCE}}/api/datastream-types/' \
        --header 'Authorization: Token {{KEY}}'

Example of JSON response:

{
    "count": XYZ,
    "next": "https://{{INSTANCE}}/api/datastream-types/?page=XZ",
    "previous": null,
    "results": [
        {
            "id": {{DATASTREAM_TYPE}},
            "name": "DATASTREAM_TYPE_NAME",
            "slug": "DATASTREAM_AUTHORIZATION",
            "api_details": {
                "base_url": "/api/v0/",
                "documentation": "",
                "metadata": [],
                "updated_at": "",
                "updated_last_30_days": false
            },
            "url": "https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/",
            "datastreams": "https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/datastreams/",
            "create_url": "https://{{INSTANCE}}/core/apidatastream/add/?ct_id={{DATASTREAM_TYPE}}",
            "logo_url": "https://{{INSTANCE}}/static/images/api/logo.svg"
        }
    ]
}

The value in an id parameter of the response is a datastream type’s ID.

Determining datastream type options#

To determine the options you can configure for a datastream type, follow these steps:

Create an OPTIONS request to the following endpoint:
```
https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/datastreams/
```
Replace {{DATASTREAM_TYPE}} with the datastream type’s ID which you acquired in the previous section.
In the HTTP request header, include the parameter Authorization with one of the following values:
- Token {{KEY}} if you use a key generated with user credentials in Management API.
- Bearer {{KEY}} if you use a key generated in the Adverity user interface.
Send the request.

As a result, you obtain the list of options you can configure for the datastream type.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request OPTIONS 'https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/datastreams/' \
        --header 'Authorization: Token {{KEY}}'

To determine the list of options you must specify for creating the datastream, search for "required": true in the JSON response.

Creating a datastream#

To send the request to create a datastream, follow these steps:

Create a POST request to the following endpoint:
```
https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/datastreams/
```
Replace {{DATASTREAM_TYPE}} with the datastream type’s ID.
In the HTTP request header, include the parameter Authorization with one of the following values:
- Token {{KEY}} if you use a key generated with user credentials in Management API.
- Bearer {{KEY}} if you use a key generated in the Adverity user interface.
In the HTTP request header, include the parameter Content-Type with value application/json.
In the HTTP request body, include the options you want to configure for the datastream as parameters.

The following parameters are always required:

datatype
Specify the datastream’s status. The value can be either "Live" or "Staging".

cron_type
The frequency of scheduled fetches. The value can be "day", for example.

name
Provide a name for the datastream.

stack
The ID of the workspace where you want to create the datastream.

The following parameters are also commonly required:

auth
The ID of the authorization you use to connect to the data source.

fields
The list of fields to collect from the data source.

report_type
The report type to collect from the data source.

time_range_preset
The time range of scheduled fetches.
Send the request.

As a result, you create the datastream which is disabled by default.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request POST 'https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/datastreams/' \
        --header 'Authorization: Token {{KEY}}' \
        --header 'Content-Type: application/json' \
        --data-raw '{
                "datatype": "Staging",
                "cron_type": "day",
                "name": "My new datastream",
                "stack": 123,
                "auth": 123
        }'

The value in the id field of the response is the newly created datastream’s ID.

Selecting authorization permissions in a datastream configuration#

There are two ways to select items, such as accounts or profiles, for which an authorization can collect data in a datastream configuration depending on the connector implementation. To determine which option should be used for a specific connector, check the options for the Datastream Type.

Using two parameters

For connectors using this option, you need to define how and which items should be selected using two parameters:

items_selection_type - select how you want to select the items from the following options:
- “all” - include all items of the type
- “include” - include only the selected items of the type
- “exclude” - include all items except the selected ones
items - enter the items to include or exclude

For example, to include data for one selected campaign, configure these parameters in the following way:

"campaign_selection_type": "include",
"campaigns": [
   "12345"
]

Using a nested parameter

For connectors using this option, you need to define how and which items should be selected using a nested parameter:

items - for the items permissions parameter, provide the following values:

value_1

Select how you want to select the items from the following options:

“all” - include all items of the type
“include” - include only the selected items of the type
“exclude” - include all items except the selected ones

value_2

Enter the items to include or exclude. If you want to include all items, enter an empty list.

For example, to include data for one selected campaign, configure the parameter in the following way:

"campaigns": [
      "include",
      [
         "12345"
      ]
]

For example, to include data for all campaigns, configure the parameter in the following way:

"campaigns": [
      "all",
      []
]

Enabling the datastream#

To enable the datastream, follow these steps:

Create a PATCH request to the following endpoint:
```
https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/
```
Replace {{DATASTREAM_ID}} with the newly created datastream’s ID.
In the HTTP request header, include the parameter Authorization with one of the following values:
- Token {{KEY}} if you use a key generated with user credentials in Management API.
- Bearer {{KEY}} if you use a key generated in the Adverity user interface.
In the HTTP request header, include the parameter Content-Type with value application/json.
In the HTTP request body, include the enabled parameter with the value true.
Send the request.

As a result, you enabled the datastream.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/' \
        --header 'Content-Type: application/json' \
        --header 'Authorization: Token {{KEY}}' \
        --data-raw '{"enabled": true}'

Creating datastreams in Management API

Contents

Creating datastreams in Management API#

Prerequisites#

Determining a datastream type’s ID#

Determining datastream type options#

Creating a datastream#

Selecting authorization permissions in a datastream configuration#

Enabling the datastream#