Managing pulling mode and smart schedule in Management API

Managing pulling mode and smart schedule in Management API#

This guide explains how to list datastreams and manage their pulling mode — switching between smart schedule and custom schedule — using the Management API.

Both schedule types coexist: switching to smart schedule does not delete your custom schedule configuration, so you can switch back at any time. For information on configuring custom fetch schedules, see Scheduling fetches.

Warning

PATCH and PUT requests accept optional query parameters to limit the scope of the operation. If no filter parameters are included, the request applies to all datastreams accessible to the API key. Always include the datastream_id parameter (or another filter) when you intend to update a specific datastream.

Note

On PATCH and PUT requests, the datastream_id and workspace_id parameters each accept at most 50 repeated values. Requests that exceed this limit return an HTTP 422 error.

Prerequisites#

Before you complete the procedures in this guide, perform all of the following actions:

Generate an API key with the appropriate scope:
- datastream:read to list datastreams.
- datastream:write to update datastream configuration.
For more information on generating API keys, see Authorizing to Management API.
Create a datastream and note its datastream ID. For more information, see Creating datastreams.

Listing datastreams#

To retrieve a list of datastreams, follow these steps:

Create a GET request to the following endpoint:
```
https://{{INSTANCE}}/api/v1/datastreams/
```
In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

Optionally, add query parameters to filter the results:

Parameter	Description
`datastream_id`	One or more datastream UUIDs. Repeat the parameter for multiple values.
`workspace_id`	One or more workspace UUIDs. Repeat the parameter for multiple values.
`workspace_scope`	Scope of the workspace filter. Accepted values: `ancestors`, `descendants`, `family`.
`enabled`	Filter by enabled state. Accepted values: `true`, `false`.
`active_pulling_mode`	Filter by active pulling mode. Accepted values: `smart`, `smart-bundle`, `schedule`.
`scheduled_deletion`	Filter by scheduled deletion state. Use `true` to return only the datastreams that are scheduled for deletion, or `false` to exclude them. Omit the parameter to return both.

Send the request.

As a result, you obtain the list of datastreams in the response.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request GET 'https://{{INSTANCE}}/api/v1/datastreams/' \
--header 'Authorization: Bearer {{KEY}}'

The example response below shows one datastream with smart schedule active:

{
  "count": 1,
  "items": [
    {
      "id": "{{DATASTREAM_UUID}}",
      "name": "{{DATASTREAM_NAME}}",
      "enabled": true,
      "workspace": {
        "id": "{{WORKSPACE_UUID}}",
        "name": "{{WORKSPACE_NAME}}"
      },
      "pulling": {
        "active_mode": "smart",
        "smart": {
          "is_active": true,
          "frequency": "daily",
          "refresh_range": {
            "value": 7,
            "unit": "day"
          },
          "deadline": null
        },
        "smart_bundle": null
      }
    }
  ]
}

The pulling.active_mode field indicates which schedule type is currently active for the datastream. Possible values are:

smart — smart schedule is active.
smart-bundle — a smart bundle schedule is active (deprecated).
schedule — a custom fetch schedule is active.
null — no schedule is configured.

Note

A datastream can have up to four independent smart-pulling ranges configured (for example, a daily check plus a wider weekly refresh). The /api/v1/ response above always shows a single frequency and refresh_range — if multiple ranges are configured, only the one with the finest cadence (the one that fires most often) is reported here; the others are not lost, but are not visible through this endpoint. To read all configured ranges, use the Listing datastreams with multiple ranges (v2) section below.

Enabling or disabling datastreams#

To enable or disable one or more datastreams, follow these steps:

Create a PATCH request to the following endpoint:
```
https://{{INSTANCE}}/api/v1/datastreams/
```
In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.
In the HTTP request header, include the parameter Content-Type with value application/json.
Optionally, add query parameters to filter which datastreams are affected. See the filter parameters described in Listing datastreams.
In the HTTP request body, include the enabled parameter set to true or false.
Send the request.

As a result, the specified datastreams are enabled or disabled. The response contains the UUIDs of all datastreams whose state actually changed.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/v1/datastreams/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{"enabled": false}'

The example response is the following:

{
  "affected": {
    "datastream_ids": ["{{DATASTREAM_UUID}}"]
  }
}

Switching to smart schedule#

To switch one or more datastreams to smart schedule, follow these steps:

Create a PATCH request to the following endpoint:
```
https://{{INSTANCE}}/api/v1/datastreams/
```
In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.
In the HTTP request header, include the parameter Content-Type with value application/json.
Optionally, add query parameters to filter which datastreams are affected. See the filter parameters described in Listing datastreams.

In the HTTP request body, include the following payload:

{
  "pulling": {
    "active_mode": "smart"
  }
}

Send the request.

As a result, smart schedule is activated for the targeted datastreams. The response contains the UUIDs of all datastreams whose pulling mode actually changed.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/v1/datastreams/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{"pulling": {"active_mode": "smart"}}'

The example response is the following:

{
  "affected": {
    "datastream_ids": ["{{DATASTREAM_UUID}}"]
  }
}

Switching to custom schedule#

To switch one or more datastreams to a custom schedule, follow these steps:

Note

A datastream must already have a custom fetch schedule configured before you can switch it to schedule mode. If any datastream in the targeted set has no fetch schedule, the request returns HTTP 400 and no changes are applied. To configure a custom schedule, see Scheduling fetches.

Create a PATCH request to the following endpoint:
```
https://{{INSTANCE}}/api/v1/datastreams/
```
In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.
In the HTTP request header, include the parameter Content-Type with value application/json.
Optionally, add query parameters to filter which datastreams are affected. See the filter parameters described in Listing datastreams.

In the HTTP request body, include the following payload:

{
  "pulling": {
    "active_mode": "schedule"
  }
}

Send the request.

As a result, custom schedule is activated for the targeted datastreams. The response contains the UUIDs of all datastreams whose pulling mode actually changed.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/v1/datastreams/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{"pulling": {"active_mode": "schedule"}}'

The example response is the following:

{
  "affected": {
    "datastream_ids": ["{{DATASTREAM_UUID}}"]
  }
}

Configuring smart-pulling settings (v1)#

Warning

This endpoint accepts only a single frequency and refresh_range. If the targeted datastreams already have multiple ranges configured (through the /api/v2/ endpoint below or through the Adverity user interface), sending this request replaces the entire configuration with the single rule you submit here — every other configured range is removed, regardless of how many there were. To add or update ranges without losing the others, use Configuring smart-pulling settings with multiple ranges (v2) instead.

To set or update the smart-pulling configuration for one or more datastreams, follow these steps:

Create a PUT request to the following endpoint:

https://{{INSTANCE}}/api/v1/datastreams/pulling/smart/

In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.
In the HTTP request header, include the parameter Content-Type with value application/json.
Optionally, add query parameters to filter which datastreams are affected. See the filter parameters described in Listing datastreams.

In the HTTP request body, include the smart-pulling configuration. The available parameters are:

Parameter	Required	Description
`is_active`	Yes	Whether smart schedule is active. Accepted values: `true`, `false`.
`frequency`	Yes	How often data is fetched. Accepted values: `daily`, `weekly`, `monthly`.
`refresh_range`	No	The rolling date window to refresh on each fetch. Omit to use the default range. Contains: `value` — integer number of units. `unit` — time unit. Accepted values: `day`, `week`, `month`.
`deadline`	No	The time by which the fetch must complete. Omit to use no deadline. Contains: `time` — time in `HH:MM:SS` format, rounded to the half hour (for example, `10:00:00` or `10:30:00`).

Send the request.

As a result, the smart-pulling configuration is applied to all targeted datastreams. The response contains the UUIDs of datastreams whose configuration actually changed.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PUT 'https://{{INSTANCE}}/api/v1/datastreams/pulling/smart/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "is_active": true,
    "frequency": "daily",
    "refresh_range": {
        "value": 7,
        "unit": "day"
    },
    "deadline": {
        "time": "06:00:00"
    }
}'

The example response is the following:

{
  "affected": {
    "datastream_ids": ["{{DATASTREAM_UUID}}"]
  }
}

Managing multiple ranges (v2)#

Management API v2 gives you access to the full smart-pulling configuration: up to four independent ranges per datastream, each with its own frequency and refresh window, read and written without losing any of the others.

Listing datastreams with multiple ranges (v2)#

To retrieve the full smart-pulling configuration for one or more datastreams, follow these steps:

Create a GET request to the following endpoint:
```
https://{{INSTANCE}}/api/v2/datastreams/
```
In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.
Optionally, add query parameters to filter the results. See the filter parameters described in Listing datastreams.
Send the request.

As a result, you obtain the list of datastreams in the response. Unlike /api/v1/, the pulling.smart object here contains the full list of configured ranges in range_spec, instead of a single frequency and refresh_range.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request GET 'https://{{INSTANCE}}/api/v2/datastreams/' \
--header 'Authorization: Bearer {{KEY}}'

The example response below shows one datastream with two smart-pulling ranges configured — a daily fetch of the last 3 days, and a weekly fetch aligned to Monday, refreshing the last 40 days:

{
  "count": 1,
  "items": [
    {
      "id": "{{DATASTREAM_UUID}}",
      "name": "{{DATASTREAM_NAME}}",
      "enabled": true,
      "workspace": {
        "id": "{{WORKSPACE_UUID}}",
        "name": "{{WORKSPACE_NAME}}"
      },
      "pulling": {
        "active_mode": "smart",
        "smart": {
          "is_active": true,
          "range_spec": [
            {
              "frequency": {"value": 1, "unit": "day"},
              "on": null,
              "range": {"value": 3, "unit": "day"}
            },
            {
              "frequency": {"value": 1, "unit": "week"},
              "on": {"day": 1, "day_type": "week"},
              "range": {"value": 40, "unit": "day"}
            }
          ],
          "deadline": null
        },
        "smart_bundle": null
      }
    }
  ]
}

Configuring smart-pulling settings with multiple ranges (v2)#

Warning

Like the /api/v1/ endpoint, this call replaces the entire range_spec for all targeted datastreams. Include every range you want to keep in the request body, not only the one you are adding or changing.

Note

The 50-value limit on repeated datastream_id and workspace_id parameters described at the top of this guide applies to /api/v1/ requests only. It does not apply to this /api/v2/ endpoint.

To set or update the smart-pulling configuration for one or more datastreams with one or more ranges, follow these steps:

Create a PUT request to the following endpoint:

https://{{INSTANCE}}/api/v2/datastreams/pulling/smart/

In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.
In the HTTP request header, include the parameter Content-Type with value application/json.
Optionally, add query parameters to filter which datastreams are affected. See the filter parameters described in Listing datastreams.

In the HTTP request body, include the smart-pulling configuration. The available parameters are:

Parameter

Required

Description

is_active

Yes

Whether smart schedule is active. Accepted values: true, false.

range_spec

Yes

A list of 1 to 4 range rules. Each rule contains:

frequency — how often this rule fetches data. Contains value (integer) and unit (day, week, or month). No two rules in the same range_spec may use the same frequency.
range — how much history to refresh each time this rule fires. Same shape as frequency. The range must cover at least one full frequency period — for example, a monthly frequency requires a range of at least 31 days; a shorter value such as {"value": 30, "unit": "day"} is rejected.
on — the day this rule aligns to. Required for a weekly or monthly frequency, and omitted (null) for a daily frequency. Contains day and day_type:
- For day_type: "week", day is the ISO day of the week, 1 (Monday) through 7 (Sunday).
- For day_type: "month", day is the day of the month, 1 through 28.

deadline

No

The time by which the fetch must complete. Omit to use no deadline. Contains:

time — time in HH:MM:SS format, rounded to the half hour (for example, 10:00:00 or 10:30:00).

Send the request.

As a result, the smart-pulling configuration is applied to all targeted datastreams. The response contains the UUIDs of datastreams whose configuration actually changed.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PUT 'https://{{INSTANCE}}/api/v2/datastreams/pulling/smart/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "is_active": true,
    "range_spec": [
        {
            "frequency": {"value": 1, "unit": "day"},
            "on": null,
            "range": {"value": 3, "unit": "day"}
        },
        {
            "frequency": {"value": 1, "unit": "week"},
            "on": {"day": 1, "day_type": "week"},
            "range": {"value": 40, "unit": "day"}
        }
    ],
    "deadline": {
        "time": "06:00:00"
    }
}'

The example response is the following:

{
  "affected": {
    "datastream_ids": ["{{DATASTREAM_UUID}}"]
  }
}

Managing pulling mode and smart schedule in Management API

Contents

Managing pulling mode and smart schedule in Management API#

Prerequisites#

Listing datastreams#

Enabling or disabling datastreams#

Switching to smart schedule#

Switching to custom schedule#

Configuring smart-pulling settings (v1)#

Managing multiple ranges (v2)#

Listing datastreams with multiple ranges (v2)#

Configuring smart-pulling settings with multiple ranges (v2)#