Managing pulling mode and smart schedule in Management API

Managing pulling mode and smart schedule in Management API#

This guide explains how to list datastreams and manage their pulling mode — switching between smart schedule and custom schedule — using the Management API.

Both schedule types coexist: switching to smart schedule does not delete your custom schedule configuration, so you can switch back at any time. For information on configuring custom fetch schedules, see Scheduling fetches.

Warning

PATCH and PUT requests accept optional query parameters to limit the scope of the operation. If no filter parameters are included, the request applies to all datastreams accessible to the API key. Always include the datastream_id parameter (or another filter) when you intend to update a specific datastream.

Note

On PATCH and PUT requests, the datastream_id and workspace_id parameters each accept at most 50 repeated values. Requests that exceed this limit return an HTTP 422 error.

Prerequisites#

Before you complete the procedures in this guide, perform all of the following actions:

Generate an API key with the appropriate scope:
- datastream:read to list datastreams.
- datastream:write to update datastream configuration.
For more information on generating API keys, see Authorizing to Management API.
Create a datastream and note its datastream ID. For more information, see Creating datastreams.

Listing datastreams#

To retrieve a list of datastreams, follow these steps:

Create a GET request to the following endpoint:
```
https://{{INSTANCE}}/api/v1/datastreams/
```
In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

Optionally, add query parameters to filter the results:

Parameter	Description
`datastream_id`	One or more datastream UUIDs. Repeat the parameter for multiple values.
`workspace_id`	One or more workspace UUIDs. Repeat the parameter for multiple values.
`workspace_scope`	Scope of the workspace filter. Accepted values: `ancestors`, `descendants`, `family`.
`enabled`	Filter by enabled state. Accepted values: `true`, `false`.
`active_pulling_mode`	Filter by active pulling mode. Accepted values: `smart`, `smart-bundle`, `schedule`.
`scheduled_deletion`	Filter by scheduled deletion state. Use `true` to return only the datastreams that are scheduled for deletion, or `false` to exclude them. Omit the parameter to return both.

Send the request.

As a result, you obtain the list of datastreams in the response.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request GET 'https://{{INSTANCE}}/api/v1/datastreams/' \
--header 'Authorization: Bearer {{KEY}}'

The example response below shows one datastream with smart schedule active:

{
  "count": 1,
  "items": [
    {
      "id": "{{DATASTREAM_UUID}}",
      "name": "{{DATASTREAM_NAME}}",
      "enabled": true,
      "workspace": {
        "id": "{{WORKSPACE_UUID}}",
        "name": "{{WORKSPACE_NAME}}"
      },
      "pulling": {
        "active_mode": "smart",
        "smart": {
          "is_active": true,
          "range_spec": [
            {
              "frequency": {"value": 1, "unit": "day"},
              "on": null,
              "range": {"value": 7, "unit": "day"}
            },
            {
              "frequency": {"value": 1, "unit": "week"},
              "on": {"day": 1, "day_type": "week"},
              "range": {"value": 30, "unit": "day"}
            }
          ],
          "deadline": null,
          "start": null,
          "frequency": "daily",
          "refresh_range": {
            "value": 6,
            "unit": "day"
          }
        },
        "smart_bundle": null
      }
    }
  ]
}

The pulling.active_mode field indicates which schedule type is currently active for the datastream. Possible values are:

smart — smart schedule is active.
smart-bundle — a smart bundle schedule is active (deprecated).
schedule — a custom fetch schedule is active.
null — no schedule is configured.

The response describes the smart schedule twice, and range_spec is the authoritative version:

range_spec lists every configured range. A datastream can have up to four, for example a daily check plus a wider weekly refresh.
frequency and refresh_range are only present to provide backwards compatibility for integrations written before range_spec existed. They summarise the same configuration as a single cadence. Adverity derives them from range_spec on every request, and reports the range with the finest cadence — the one that fires most often — as frequency, with the history it refreshes beyond one cadence period as refresh_range, always in days. In the example above, the weekly range does not appear in this summary.

Enabling or disabling datastreams#

To enable or disable one or more datastreams, follow these steps:

Create a PATCH request to the following endpoint:
```
https://{{INSTANCE}}/api/v1/datastreams/
```
In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.
In the HTTP request header, include the parameter Content-Type with value application/json.
Optionally, add query parameters to filter which datastreams are affected. See the filter parameters described in Listing datastreams.
In the HTTP request body, include the enabled parameter set to true or false.
Send the request.

As a result, the specified datastreams are enabled or disabled. The response contains the UUIDs of all datastreams whose state actually changed.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/v1/datastreams/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{"enabled": false}'

The example response is the following:

{
  "affected": {
    "datastream_ids": ["{{DATASTREAM_UUID}}"]
  }
}

Switching to smart schedule#

To switch one or more datastreams to smart schedule, follow these steps:

Create a PATCH request to the following endpoint:
```
https://{{INSTANCE}}/api/v1/datastreams/
```
In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.
In the HTTP request header, include the parameter Content-Type with value application/json.
Optionally, add query parameters to filter which datastreams are affected. See the filter parameters described in Listing datastreams.

In the HTTP request body, include the following payload:

{
  "pulling": {
    "active_mode": "smart"
  }
}

Send the request.

As a result, smart schedule is activated for the targeted datastreams. The response contains the UUIDs of all datastreams whose pulling mode actually changed.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/v1/datastreams/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{"pulling": {"active_mode": "smart"}}'

The example response is the following:

{
  "affected": {
    "datastream_ids": ["{{DATASTREAM_UUID}}"]
  }
}

Switching to custom schedule#

To switch one or more datastreams to a custom schedule, follow these steps:

Note

A datastream must already have a custom fetch schedule configured before you can switch it to schedule mode. If any datastream in the targeted set has no fetch schedule, the request returns HTTP 400 and no changes are applied. To configure a custom schedule, see Scheduling fetches.

Create a PATCH request to the following endpoint:
```
https://{{INSTANCE}}/api/v1/datastreams/
```
In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.
In the HTTP request header, include the parameter Content-Type with value application/json.
Optionally, add query parameters to filter which datastreams are affected. See the filter parameters described in Listing datastreams.

In the HTTP request body, include the following payload:

{
  "pulling": {
    "active_mode": "schedule"
  }
}

Send the request.

As a result, custom schedule is activated for the targeted datastreams. The response contains the UUIDs of all datastreams whose pulling mode actually changed.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/v1/datastreams/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{"pulling": {"active_mode": "schedule"}}'

The example response is the following:

{
  "affected": {
    "datastream_ids": ["{{DATASTREAM_UUID}}"]
  }
}

Configuring smart-pulling settings#

Warning

This endpoint replaces the entire configuration. The range_spec in the request body carries the full list of ranges, so any range you leave out is removed from the targeted datastreams. To change one range, send the complete range_spec with that range altered.

The body accepts only the parameters listed below. The Listing datastreams response also carries a frequency and refresh_range pair, but those are read-only and are not part of the PUT request below; including them returns a validation error.

To set or update the smart-pulling configuration for one or more datastreams, follow these steps:

Create a PUT request to the following endpoint:

https://{{INSTANCE}}/api/v1/datastreams/pulling/smart/

In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.
In the HTTP request header, include the parameter Content-Type with value application/json.
Optionally, add query parameters to filter which datastreams are affected. See the filter parameters described in Listing datastreams.

In the HTTP request body, include the smart-pulling configuration. The available parameters are:

Parameter	Required	Description
`is_active`	Yes	Whether smart schedule is active. Accepted values: `true`, `false`.
`range_spec`	Yes	The ranges to fetch, as a list of one to four objects. Each range fires on its own cadence and refreshes its own window of history. Every range in the list must use a different `frequency`. Each object contains: `frequency` — how often this range is fetched. Contains `value`, an integer, and `unit`. Only five combinations are accepted: every `1` `day`, every `1` `week`, every `2` `week`, every `4` `week`, and every `1` `month`. Anything else is rejected. `range` — the rolling window of history to refresh each time this range fires. Contains `value`, an integer, and `unit`. The window must cover at least one `frequency` period, and must be one of the time ranges the data source offers. `on` — the day the range fires on. Required for a `week` or `month` frequency, and not allowed for a `day` frequency. Contains `day`, an integer from 1 to 7 for a week (Monday is 1) or 1 to 28 for a month, and `day_type`, either `week` or `month`, matching the frequency unit.
`deadline`	No	The time by which the fetch must complete. Omit or send `null` to inherit the workspace’s default deadline (which may itself be no deadline). Contains: `time` — time in `HH:MM:SS` format, rounded to the half hour (for example, `10:00:00` or `10:30:00`) and in UTC. An offset is rejected rather than converted.
`start`	No	The earliest time of day a fetch may start. Uses the same `HH:MM:SS` format, half-hour rounding, and UTC-only rule as `deadline`. Contains: `time` — the earliest start time. Unlike `deadline`, `start` has four states, and omitting it is not the same as sending it as `null`: Omit the field entirely to leave the datastream’s current start setting unchanged. Send `start` as `null` to remove any per-datastream start time and inherit the workspace’s default instead. Send `start` with `time` set to `null` to lift the restriction outright. This is an explicit per-datastream setting, not inheritance: it overrides the workspace default rather than falling back to it. Send `start` with a `time` value to set a specific start time for this datastream, overriding the workspace default. `start` must resolve to a time earlier than the effective `deadline`, once workspace defaults are resolved for any value that is inherited. Sending a `start` at or after the deadline fails with a `400` error.

Send the request.

As a result, the smart-pulling configuration is applied to all targeted datastreams. The response contains the UUIDs of datastreams whose configuration actually changed.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PUT 'https://{{INSTANCE}}/api/v1/datastreams/pulling/smart/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "is_active": true,
    "range_spec": [
        {
            "frequency": {"value": 1, "unit": "day"},
            "on": null,
            "range": {"value": 7, "unit": "day"}
        },
        {
            "frequency": {"value": 1, "unit": "week"},
            "on": {"day": 1, "day_type": "week"},
            "range": {"value": 30, "unit": "day"}
        }
    ],
    "deadline": {
        "time": "06:00:00"
    },
    "start": {
        "time": "22:00:00"
    }
}'

This example configures two ranges: the last 7 days refreshed every day, and the last 30 days refreshed every Monday.

The example response is the following:

{
  "affected": {
    "datastream_ids": ["{{DATASTREAM_UUID}}"]
  }
}

Managing pulling mode and smart schedule in Management API

Contents

Managing pulling mode and smart schedule in Management API#

Prerequisites#

Listing datastreams#

Enabling or disabling datastreams#

Switching to smart schedule#

Switching to custom schedule#

Configuring smart-pulling settings#