Creating datastreams in Management API

This article explains how to create datastreams with the Management API.

Determining a Datastream Type's ID

To determine a Datastream Type's ID, follow these steps:

1. Create a GET request to the following endpoint:

   https://{{INSTANCE}}/api/datastream-types/

2. In the HTTP request header, include the parameter Authorization with one of the following values:

   • Token {{KEY}} if you use a key generated with user credentials in Management API.

   • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

3. Send the request.

As a result, you obtain the available Datastream Types and their IDs in the response.

https://{{INSTANCE}}/api/datastream-types/?search={{DATASTREAM_TYPE_NAME}}

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request GET 'https://{{INSTANCE}}/api/datastream-types/' \
		--header 'Authorization: Token {{KEY}}'

Example of JSON response:

{
    "count": XYZ,
    "next": "https://{{INSTANCE}}/api/datastream-types/?page=XZ",
    "previous": null,
    "results": [
        {
            "id": {{DATASTREAM_TYPE}},
            "name": "DATASTREAM_TYPE_NAME",
            "slug": "DATASTREAM_AUTHORIZATION",
			"api_details": {
				"base_url": "/api/v0/",
				"documentation": "",
				"metadata": [],
				"updated_at": "",
				"updated_last_30_days": false
			},
            "url": "https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/",
            "datastreams": "https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/datastreams/",
            "create_url": "https://{{INSTANCE}}/core/apidatastream/add/?ct_id={{DATASTREAM_TYPE}}",
            "logo_url": "https://{{INSTANCE}}/static/images/api/logo.svg"
        }
    ]
}

The value in an id parameter of the response is a Datastream Type's ID.

Determining Datastream Type options

To determine the options you can configure for a Datastream Type, follow these steps:

1. Create an OPTIONS request to the following endpoint:

   https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/datastreams/

   Replace {{DATASTREAM_TYPE}} with the Datastream Type's ID which you acquired in the previous section.

2. In the HTTP request header, include the parameter Authorization with one of the following values:

   • Token {{KEY}} if you use a key generated with user credentials in Management API.

   • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

3. Send the request.

As a result, you obtain the list of options you can configure for the Datastream Type.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request OPTIONS 'https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/datastreams/' \
		--header 'Authorization: Token {{KEY}}'

To determine the list of options you must specify for creating the datastream, search for "required": true in the JSON response.

Creating a datastream

To send the request to create a datastream, follow these steps:

1. Create a POST request to the following endpoint:

   https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/datastreams/

   Replace {{DATASTREAM_TYPE}} with the Datastream Type's ID.

2. In the HTTP request header, include the parameter Authorization with one of the following values:

   • Token {{KEY}} if you use a key generated with user credentials in Management API.

   • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

3. In the HTTP request header, include the parameter Content-Type with value application/json.

4. In the HTTP request body, include the options you want to configure for the datastream as parameters.

   The following parameters are always required:

   datatype

   Specify the datastream's status. The value can be either "Live" or "Staging".

   cron_type

   The frequency of scheduled fetches. The value can be "day", for example.

   name

   Provide a name for the datastream.

   stack

   The ID of the workspace where you want to create the datastream.

   The following parameters are also commonly required:

   auth

   The ID of the authorization you use to connect to the data source.

   fields

   The list of fields to collect from the data source.

   report_type

   The report type to collect from the data source.

   time_range_preset

   The time range of scheduled fetches.

5. Send the request.

As a result, you create the datastream which is disabled by default.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request POST 'https://{{INSTANCE}}/api/datastream-types/{{DATASTREAM_TYPE}}/datastreams/' \
		--header 'Authorization: Token {{KEY}}' \
		--header 'Content-Type: application/json' \
		--data-raw '{
				"datatype": "Staging",
				"cron_type": "day",
				"name": "My new datastream",
				"stack": 123,
				"auth": 123
		}'

The value in the id field of the response is the newly created datastream's ID.

Selecting authorization permissions in a datastream configuration

There are two ways to select items, such as accounts or profiles, for which an authorization can collect data in a datastream configuration depending on the connector implementation. To determine which option should be used for a specific connector, check the options for the Datastream Type.

Using two parameters

For connectors using this option, you need to define how and which items should be selected using two parameters:

• items_selection_type - select how you want to select the items from the following options:

  • "all" - include all items of the type

  • "include" - include only the selected items of the type

  • "exclude" - include all items except the selected ones

• items - enter the items to include or exclude

For example, to include data for one selected campaign, configure these parameters in the following way:

"campaign_selection_type": "include",
"campaigns": [
		"12345"
]

Using a nested parameter

For connectors using this option, you need to define how and which items should be selected using a nested parameter:

• items - for the items permissions parameter, provide the following values:

  value_1

  Select how you want to select the items from the following options:

  • "all" - include all items of the type

  • "include" - include only the selected items of the type

  • "exclude" - include all items except the selected ones

  value_2

  Enter the items to include or exclude. If you want to include all items, enter an empty list.

For example, to include data for one selected campaign, configure the parameter in the following way:

"campaigns": [
	"include",
	[
		"12345"
	]
]

For example, to include data for all campaigns, configure the parameter in the following way:

"campaigns": [
	"all",
	[]
]

Enabling the datastream

To enable the datastream, follow these steps:

1. Create a PATCH request to the following endpoint:

   https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/

   Replace {{DATASTREAM_ID}} with the newly created datastream's ID.

2. In the HTTP request header, include the parameter Authorization with one of the following values:

   • Token {{KEY}} if you use a key generated with user credentials in Management API.

   • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

3. In the HTTP request header, include the parameter Content-Type with value application/json.

4. In the HTTP request body, include the enabled parameter with the value true.

5. Send the request.

As a result, you enabled the datastream.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/' \
		--header 'Content-Type: application/json' \
		--header 'Authorization: Token {{KEY}}' \
		--data-raw '{"enabled": true}'