Scheduling fetches in Management API

Scheduling fetches in Management API#

This guide explains how to configure custom cron-based fetch schedules for your datastreams using the Management API.

Note

Adverity also supports smart schedule, which automatically determines the optimal fetch frequency based on your data patterns. Smart schedule is managed through a separate set of API endpoints. For more information, see Managing pulling mode and smart schedule.

Introduction#

Scheduling lets you configure fetches that periodically retrieve data extracts from your datastreams.

Prerequisites#

Before you complete the procedure in this guide, perform all of the following actions:

  • Create a datastream and determine its datastream ID. For more information, see Creating datastreams.

Retrieving existing schedules#

To retrieve existing schedules, follow these steps:

  1. Create a GET request to the following endpoint:

    https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/schedules

  2. In the HTTP request header, include the parameter Authorization with one of the following values:

    • Token {{KEY}} if you use a key generated with user credentials in Management API.

    • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

  3. Send the request.

  4. In the response, search for schedules to find currently configured schedules.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/schedules' \
--header 'Authorization: Token {{KEY}}'

Retrieving schedule and date range options#

To retrieve schedule options and date range options for a particular datastream, follow these steps:

  1. Create an OPTIONS request to the following endpoint:

    https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/

  2. In the HTTP request header, include the parameter Authorization with one of the following values:

    • Token {{KEY}} if you use a key generated with user credentials in Management API.

    • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

  3. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request OPTIONS 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/' \
--header 'Authorization: Token {{KEY}}'

Example of the part of response with two options for fetch date range (choices) of the time_range_preset variable:

"time_range_preset": {
   "type": "choice",
   "required": false,
   "read_only": false,
   "label": "Time range preset",
   "help_text": "Select one of the time range presets.",
   "choices": [
        {
          "display_name": "Fixed Time Range",
          "value": 0
        },
        {
          "display_name": "Yesterday",
          "value": 13
        },

Configuring a schedule and date range#

After you configured an OPTIONS call to retrieve the accepted parameters for schedule options, you can edit these parameters. To configure a schedule, follow these steps:

  1. Create a PATCH request to the following endpoint:

    https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/

  2. In the HTTP request header, include the parameter Authorization with one of the following values:

    • Token {{KEY}} if you use a key generated with user credentials in Management API.

    • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

  3. In the HTTP request header, include the parameter Content-Type with value application/json.

  4. In the HTTP request body, include the parameter time_range_preset. Set the value of this parameter according to the options retrieved in the API call described in the previous section. This parameter specifies the date range of the fetch (for which days, weeks, and other time specific values you are requesting the data).

  5. In the HTTP request body, include the parameter and value "cron_type": "week". This parameter specifies how often the fetch is periodically scheduled. "cron_type": "week" fetches the data of a datastream once per week.

  6. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/' \
--header 'Authorization: Token {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "time_range_preset": 11,
    "cron_type": "week"
}'

Creating a new schedule#

To create a new schedule, send a PATCH request that includes all required variables for a fetch schedule. Find the necessary values using the OPTIONS request described in section Retrieving schedule options above. To create a new schedule, follow these steps:

  1. Create a PATCH request to the following endpoint:

    https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/

  2. In the HTTP request header, include the parameter Authorization with one of the following values:

    • Token {{KEY}} if you use a key generated with user credentials in Management API.

    • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

  3. In the HTTP request header, include the parameter Content-Type with value application/json.

  4. In the HTTP request body, include the following parameters and values wrapped in a schedules array (edit them as necessary):

    {"schedules": [
    {
        "cron_preset": "CRON_EVERY_DAY",
        "cron_type": "day",
        "cron_interval": 1,
        "cron_start_of_day": "00:00:00",
        "cron_interval_start": 1,
        "time_range_preset": 0,
        "time_range_preset_label": "Fixed Time Range",
        "delta_type": 1,
        "delta_interval": 1,
        "delta_start_of_day": "00:00:00",
        "delta_interval_start": 1,
        "fixed_start": "2021-12-01",
        "fixed_end": "2021-12-02",
        "offset_days": 0
    }
]}

Note

Schedule fields must be passed inside the schedules array. Sending these fields at the top level of the request body has no effect.

  1. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/' \
--header 'Authorization: Token {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{"schedules": [
{
    "cron_preset": "CRON_EVERY_DAY",
    "cron_type": "day",
    "cron_interval": 1,
    "cron_start_of_day": "00:00:00",
    "cron_interval_start": 1,
    "time_range_preset": 0,
    "time_range_preset_label": "Fixed Time Range",
    "delta_type": 1,
    "delta_interval": 1,
    "delta_start_of_day": "00:00:00",
    "delta_interval_start": 1,
    "fixed_start": "2021-12-01",
    "fixed_end": "2021-12-02",
    "offset_days": 0
}
]}'

Note

The delta_type, delta_interval, delta_start_of_day, and delta_interval_start fields in the example are legacy parameters for a time_range_preset that is no longer available. You can omit them from new schedules — they are accepted but have no effect. Existing integrations that already send these fields are unaffected.

Creating multiple schedules#

Similarly to the procedure described in the previous section, send a PATCH request that includes all required variables for multiple fetch schedules. Find the necessary values using the OPTIONS request described in section Retrieving schedule options above. To create multiple schedules, follow these steps:

  1. Create a PATCH request to the following endpoint:

    https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/

  2. In the HTTP request header, include the parameter Authorization with one of the following values:

    • Token {{KEY}} if you use a key generated with user credentials in Management API.

    • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

  3. In the HTTP request header, include the parameter Content-Type with value application/json.

  4. In the HTTP request body, include the following parameters and values (edit them as necessary):

    {
    "cron_preset": "CRON_EVERY_DAY",
    "cron_type": "day",
    "cron_interval": 1,
    "cron_start_of_day": "00:00:00",
    "cron_interval_start": 1,
    "time_range_preset": 0,
    "time_range_preset_label": "Fixed Time Range",
    "delta_type": 1,
    "delta_interval": 1,
    "delta_start_of_day": "00:00:00",
    "delta_interval_start": 1,
    "fixed_start": "2021-12-01",
    "fixed_end": "2021-12-02",
    "offset_days":
},
{
    "cron_preset": "CRON_EVERY_WEEK",
    "cron_type": "week",
    "cron_interval": 1,
    "cron_start_of_day": "00:00:00",
    "cron_interval_start": 1,
    "time_range_preset": 0,
    "time_range_preset_label": "Fixed Time Range",
    "delta_type": 1,
    "delta_interval": 1,
    "delta_start_of_day": "00:00:00",
    "delta_interval_start": 1,
    "fixed_start": "2021-12-01",
    "fixed_end": "2021-12-02",
    "offset_days": 0
}

  5. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/' \
--header 'Authorization: Token {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{"schedules": [
{
    "cron_preset": "CRON_EVERY_DAY",
    "cron_type": "day",
    "cron_interval": 1,
    "cron_start_of_day": "00:00:00",
    "cron_interval_start": 1,
    "time_range_preset": 0,
    "time_range_preset_label": "Fixed Time Range",
    "delta_type": 1,
    "delta_interval": 1,
    "delta_start_of_day": "00:00:00",
    "delta_interval_start": 1,
    "fixed_start": "2021-12-01",
    "fixed_end": "2021-12-02",
    "offset_days": 0
},
{
    "cron_preset": "CRON_EVERY_WEEK",
    "cron_type": "week",
    "cron_interval": 1,
    "cron_start_of_day": "00:00:00",
    "cron_interval_start": 1,
    "time_range_preset": 0,
    "time_range_preset_label": "Fixed Time Range",
    "delta_type": 1,
    "delta_interval": 1,
    "delta_start_of_day": "00:00:00",
    "delta_interval_start": 1,
    "fixed_start": "2021-12-01",
    "fixed_end": "2021-12-02",
    "offset_days": 0
}
]
}'

Note

The delta_type, delta_interval, delta_start_of_day, and delta_interval_start fields in the example are legacy parameters for a time_range_preset that is no longer available. You can omit them from new schedules — they are accepted but have no effect. Existing integrations that already send these fields are unaffected.

Using smart schedule#

Smart Schedule is an automated scheduling mode that handles fetch timing intelligently. Instead of defining a fixed cron expression, Smart Schedule automatically determines when to fetch data for each datastream, taking into account factors such as API rate limits, datastream dependencies, and historical processing times. For more information about Smart Schedule, see Using Smart Schedule.

You can switch a datastream between smart schedule and custom (cron-based) schedule using PATCH /api/v1/datastreams/ with the datastream_id query parameter.

Note

The /api/v1/datastreams/ endpoint requires a key generated in the Adverity user interface. Use Bearer {{KEY}} in the Authorization header. For more information, see Generating an API key in Adverity.

Warning

Always include the datastream_id query parameter. If it is omitted, the PATCH request applies to all datastreams accessible to the API key.

Enabling smart schedule#

To enable smart schedule on a datastream, follow these steps:

  1. Create a PATCH request to the following endpoint, replacing {{DATASTREAM_UUID}} with the UUID of the datastream:

    https://{{INSTANCE}}/api/v1/datastreams/?datastream_id={{DATASTREAM_UUID}}

  2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

  3. In the HTTP request header, include the parameter Content-Type with value application/json.

  4. In the HTTP request body, set pulling.active_mode to "smart":

    {
    "pulling": {
        "active_mode": "smart"
    }
}

  5. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/v1/datastreams/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{"pulling": {"active_mode": "smart"}}'

A successful request returns a 200 response listing the affected datastreams:

{
    "affected": {
        "datastream_ids": ["{{DATASTREAM_UUID}}"]
    }
}

Switching back to a custom schedule#

To switch a datastream back to a custom (cron-based) schedule, follow these steps:

  1. Create a PATCH request to the following endpoint, replacing {{DATASTREAM_UUID}} with the UUID of the datastream:

    https://{{INSTANCE}}/api/v1/datastreams/?datastream_id={{DATASTREAM_UUID}}

  2. In the HTTP request header, include the parameter Authorization with value Bearer {{KEY}}.

  3. In the HTTP request header, include the parameter Content-Type with value application/json.

  4. In the HTTP request body, set pulling.active_mode to "schedule":

    {
    "pulling": {
        "active_mode": "schedule"
    }
}

  5. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request PATCH 'https://{{INSTANCE}}/api/v1/datastreams/?datastream_id={{DATASTREAM_UUID}}' \
--header 'Authorization: Bearer {{KEY}}' \
--header 'Content-Type: application/json' \
--data-raw '{"pulling": {"active_mode": "schedule"}}'

After switching back to a custom schedule, configure the cron fields (such as cron_type, cron_interval, and cron_start_of_day) using the procedure described in Configuring a schedule and date range.

Note

The datastream UUID is available from the GET /api/v1/datastreams/ endpoint. Note that this is the UUID-based identifier used by the /api/v1/ endpoints, not the integer ID used by the legacy API.

Filtering datastreams by schedule type#

The GET /api/v1/datastreams/ endpoint supports filtering by active pulling mode. Use the active_pulling_mode query parameter with a value of "smart" or "schedule" to list only datastreams using that mode. For the full list of filter parameters, refer to the interactive API documentation at https://{{INSTANCE}}/api/v1/docs/.

Contents