Listing datastreams in Management API

This guide explains how to list datastreams with the v1 endpoint GET /api/v1/datastreams/, which filters on and returns the following:

• datastream UUIDs

• workspaces

• enabled state

• pulling mode and schedules

• tags

The v1 endpoints follow their own conventions for authentication, identifiers, pagination, filters, and errors. For more information, see Working with v1 endpoints.

Prerequisites

Before you complete the procedures in this guide, perform the following action:

• Use a Connect API key with Datastream: Read-only (datastream:read). Datastream: Write also works. Without this permission, Management API returns HTTP 403. You cannot change the permissions of a key, so to add one, create a new key. For more information, see Creating a Connect API key in Adverity.

Listing datastreams

To list datastreams, follow these steps:

1. Create a GET request to the following endpoint:

   https://{{INSTANCE}}/api/v1/datastreams/
   

2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

3. Optionally, add query parameters to filter the results. If you combine several parameters, the response contains only the datastreams that match all of them.

   Parameter	Description
   datastream_id	Only the datastreams with these UUIDs. Repeat the parameter for several values, up to 50.
   workspace_id	Only the datastreams that belong directly to these workspaces. datastreams in child workspaces are not included. Repeat the parameter for several values, up to 50.
   workspace_scope	ancestors, descendants, or family: only the datastreams in the workspace given by scope_from_workspace_id and the workspaces above it, below it, or both.
   scope_from_workspace_id	The workspace UUID that workspace_scope starts from. Required when the instance has more than one top-level workspace.
   enabled	true for enabled datastreams only, false for disabled datastreams only.
   active_pulling_mode	Only the datastreams whose active pulling mode is smart, smart-bundle, or schedule.
   scheduled_deletion	true for datastreams scheduled for deletion only, false to exclude them. Omit the parameter to return both.
   tag	Only the datastreams with a tag of this name, ignoring case. The name matches in every workspace the key can see. To match the tag of one workspace only, add workspace_id. Repeat the parameter for several names, up to 50. For more information, see Tagging datastreams in Management API.
   tag_match	any (default) for the datastreams with at least one of the given tags, all for the datastreams with every given tag.
   page	The page number, starting at 1. Default: 1.
   page_size	The number of items per page. Default: 50. Values above 100 return 100 items.

4. Send the request.

As a result, you obtain the datastreams that the key can see, in the order they were created.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/v1/datastreams/?enabled=true&page_size=100' \
--header 'Authorization: Bearer {{KEY}}'

The example response below shows one datastream with a custom fetch schedule:

{
  "items": [
    {
      "id": "{{DATASTREAM_UUID}}",
      "name": "{{DATASTREAM_NAME}}",
      "enabled": true,
      "workspace": {
        "id": "{{WORKSPACE_UUID}}",
        "name": "{{WORKSPACE_NAME}}"
      },
      "tags": [
        {
          "id": "{{TAG_UUID}}",
          "name": "prod"
        }
      ],
      "pulling": {
        "active_mode": "schedule",
        "smart": null,
        "smart_bundle": null,
        "schedule": [
          {
            "id": 4821,
            "cron_type": "day",
            "cron_interval": 1,
            "cron_interval_start": 1,
            "cron_start_of_day": "03:00:00",
            "cron_preset": "CRON_EVERY_DAY",
            "time_range_preset": 4,
            "time_range_preset_label": "Last 7 Days",
            "delta_type": "day",
            "delta_interval": 1,
            "delta_interval_start": 1,
            "delta_start_of_day": "00:00:00",
            "offset_days": 0,
            "fixed_start": null,
            "fixed_end": null,
            "not_before_date": null,
            "not_before_time": null
          }
        ]
      }
    }
  ],
  "count": 1
}

Response fields

Each item in items has the following fields:

Field	Description
id	The datastream UUID. Use it wherever a v1 endpoint asks for a datastream_id.
name	The datastream name.
enabled	true if the datastream is enabled.
workspace.id	The UUID of the workspace the datastream belongs to.
workspace.name	The name of the workspace the datastream belongs to.
tags	The tags of the datastream, each with its id (UUID) and name. An empty list if it has none. For more information, see Tagging datastreams in Management API.
pulling.active_mode	The schedule type that is active: smart (smart schedule), smart-bundle (smart bundle schedule, deprecated), schedule (custom fetch schedule), or null (no schedule is configured).
pulling.smart	The smart schedule configuration, or null if the datastream has none. For more information, see Managing pulling mode and smart schedule in Management API.
pulling.smart_bundle	The smart bundle schedule entries, or null if the datastream has none. Deprecated.
pulling.schedule	The custom fetch schedule entries, or null if the datastream has none. Each entry has the same fields as a fetch schedule of the legacy endpoints, plus a numeric id. For more information, see Scheduling fetches in Management API.

A datastream can have several schedule configurations stored at once, for example a custom fetch schedule it used before switching to smart schedule. pulling.active_mode tells you which one runs.

To read fields that this endpoint does not return, such as the connector configuration, use the legacy endpoint GET /api/datastreams/{{DATASTREAM_ID}}/. For more information, see Configuring datastreams in Management API.