Managing jobs in Management API

Managing jobs in Management API#

This article documents how to list current jobs, list past jobs, obtain a status of jobs in progress, and how to stop jobs with the Management API.

Listing jobs#

Listing all jobs in an instance#

Use the API call documented in this section to list all currently running and scheduled jobs (for example, fetch jobs) in a whole instance.

To list all running and scheduled jobs, follow these steps:

  1. Create a GET request to the following endpoint:

    https://{{INSTANCE}}/api/jobs

  2. In the HTTP request header, include the parameter Authorization with one of the following values:

    • Token {{KEY}} if you use a key generated with user credentials in Management API.

    • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

  3. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location --request GET 'https://{{INSTANCE}}/api/jobs/' \
--header 'Authorization: Token {{KEY}}'

The response to this request includes the state_label parameter with one of the following values assigned:

  • Scheduled

  • Running

  • Success

  • Cancelled

  • Failed

  • Discarded - this status is set in the following cases:

    • If a preceding job failed or was cancelled, the follow up jobs are discarded.

    • If a job does not start running within 2 days of the fetch start, it is discarded.

    • If a job is cancelled manually, it is discarded.

  • Stale - this status is not set currently. Generally, it can be used to label jobs that will not complete due to internal failures and lost or removed tasks.

The response of a fetch job also includes the user field:

Field

Description

user

The email of the user who triggered the fetch. This field is null for system-triggered fetches, such as scheduled, smart schedule, or backfill fetches.

Note

Adverity records the triggering user only for fetches triggered after this feature was released. Fetches triggered through the Management API before the release have no user recorded, and this information cannot be reconstructed retroactively.

Filtering the jobs#

You can filter the jobs returned from this endpoint with a number of parameters:

Parameter

Value

Description

datastream_id

{{DATASTREAM_ID}}

Use this parameter to list the jobs of a specific datastream.

start and end

Dates YYYY-MM-DD or timestamps YYYY-MM-DDT00:00:00Z

Use these parameters to list the jobs within a specific date or time range.

type

One of the options: Fetch, Enrich, Transfer

Use this parameter to list the jobs of a specific type.

user

An email address, for example jane@doe.com

Use this parameter to list only the jobs triggered by a specific user.

triggered_by_user

true or false

Use this parameter to list only human-triggered jobs (true) or only system-triggered jobs (false).

For example, to list the jobs of a datastream, import the following request example as raw text to your HTTP client (such as Postman):

curl --location -g --request GET 'https://{{INSTANCE}}/api/jobs/?datastream_id={{DATASTREAM_ID}}/' \
--header 'Authorization: Token {{KEY}}'

For example, to list the jobs within a given date range, import the following request example as raw text to your HTTP client (such as Postman):

curl --location -g --request GET 'https://{{INSTANCE}}/api/jobs/?start=YYYY-MM-DD&end=YYYY-MM-DD' \
--header 'Authorization: Token {{KEY}}'

For example, to list the jobs within a more specific time rangeby using timestamps, import the following request example as raw text to your HTTP client (such as Postman):

curl --location -g --request GET 'https://{{INSTANCE}}/api/jobs/?start=YYYY-MM-DDT00:00:00Z&end=YYYY-MM-DDT00:00:00Z' \
--header 'Authorization: Token {{KEY}}'

For example, to list the jobs of a certain type, for example Fetch, import the following request example as raw text to your HTTP client (such as Postman):

curl --location -g --request GET 'https://{{INSTANCE}}/api/jobs/?type=Fetch' \
--header 'Authorization: Token {{KEY}}'

For example, to list only the jobs triggered by a specific user, import the following request example as raw text to your HTTP client (such as Postman):

curl --location -g --request GET 'https://{{INSTANCE}}/api/jobs/?user=jane@doe.com' \
--header 'Authorization: Token {{KEY}}'

For example, to list only the jobs triggered by a person rather than by the system, import the following request example as raw text to your HTTP client (such as Postman):

curl --location -g --request GET 'https://{{INSTANCE}}/api/jobs/?triggered_by_user=true' \
--header 'Authorization: Token {{KEY}}'

Listing the fetch jobs of a specific datastream#

To list the fetch jobs of a single datastream, you can also use the dedicated /api/datastreams/{{DATASTREAM_ID}}/jobs/ endpoint. This endpoint returns a paginated list of the datastream’s fetch jobs, ordered from the most recent to the oldest.

To list the fetch jobs of a datastream, follow these steps:

  1. Create a GET request to the following endpoint:

    https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/jobs/

    Replace {{DATASTREAM_ID}} with the datastream’s ID.

  2. In the HTTP request header, include the parameter Authorization with one of the following values:

    • Token {{KEY}} if you use a key generated with user credentials in Management API.

    • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

  3. Send the request.

As a result, you obtain a paginated list of the datastream’s fetch jobs.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/jobs/' \
--header 'Authorization: Token {{KEY}}'

The response is a paginated envelope with the count, next, previous, and results fields. Use the page and page_size query parameters to page through the results:

curl --location -g --request GET 'https://{{INSTANCE}}/api/datastreams/{{DATASTREAM_ID}}/jobs/?page=1&page_size=50' \
--header 'Authorization: Token {{KEY}}'

Note

This endpoint returns fetch jobs only and does not accept the filter parameters described in Filtering the jobs. To filter by job type, user, or date range, or to list other job types (such as Enrich or Transfer), use GET /api/jobs/?datastream_id={{DATASTREAM_ID}} instead.

Obtaining the status of a job in progress#

To obtain a status of a job in progress, follow these steps:

  1. Create a GET request to the following endpoint:

    https://{{INSTANCE}}/api/jobs/{{JOB_ID}}/imported/

    Find the value of JOB_ID parameter in the response of the API request described in the previous section. In this response, JOB_ID parameter is named as id.

  2. In the HTTP request header, include the parameter Authorization with one of the following values:

    • Token {{KEY}} if you use a key generated with user credentials in Management API.

    • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

  3. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request GET 'https://{{INSTANCE}}/api/jobs/{{JOB_ID}}/imported/' \
--header 'Authorization: Token {{KEY}}'

The response contains the following fields:

Field

Description

imported

Boolean. true if the job’s data has successfully landed.

completed

Boolean. true when the job lifecycle is finished, regardless of whether data landed. Use this field to detect completion when imported may remain false, for example when extracts were discarded during an Enrich cascade.

Note

Poll this endpoint until completed is true, then check imported to determine whether data landed successfully.

Stopping jobs#

To stop a job, list the currently running jobs, and find the JOB_ID of the job to stop. You can only stop jobs in the fetch stage.

To stop a job using JOB_ID, follow these steps:

  1. Create a POST request to the following endpoint:

    https://{{INSTANCE}}/api/jobs/{{JOB_ID}}/stop/

  2. In the HTTP request header, include the parameter Authorization with one of the following values:

    • Token {{KEY}} if you use a key generated with user credentials in Management API.

    • Bearer {{KEY}} if you use a key generated in the Adverity user interface.

  3. Send the request.

Import the request example as raw text to your HTTP client (such as Postman). The cURL request example is the following:

curl --location -g --request POST 'https://{{INSTANCE}}/api/jobs/{{JOB_ID}}/stop/' \
--header 'Authorization: Token {{KEY}}'
Contents