Introduction to the Activity page

Introduction

A task is a single process that you perform using one datastream. It always starts with a fetch and can include enriching, storing, and loading the collected data. A task consists of three stages:

  • Fetch

  • (Optional) Transform

  • (Optional) Load

You can see all tasks performed using a datastream in the datastream overview, or see all the tasks performed in your current workspace in the Activity page.

The Activity page displays an overview of tasks from collecting data to loading data into Adverity Data Storage and destinations. This is called the task monitor.

With the task monitor, you can achieve the following aims:

  • Receive a high-level overview of your account's activity.

  • See errors in the pipeline and retry failed tasks.

  • Display detailed data for each task.

  • Understand the status of your data batch.

  • See how much time each task in the pipeline took.

Viewing the task monitor

To view the task monitor, follow these steps:

  1. Go to the Activity page.

The three boxes at the top of the page are the task monitor Key Performance Indicator (KPI) boxes.

Successful tasks

The Successful tasks (today) KPI box displays the following:

  • The number of successful tasks today.

  • The number of successful tasks compared to yesterday.

This KPI box includes scheduled and manual fetches. Days are based on the user's time zone.

New failed tasks

The New failed tasks (today) KPI box displays the number of tasks that have failed today because of raised errors.

You can acknowledge a failed task, so it is no longer displayed in the Failed tab. Acknowledging a failed task does not affect the numbers in the New failed tasks (today) KPI box. For more information, see Acknowledging a task with issues.

Running tasks

The Running tasks KPI box displays the number of tasks that are currently running.

Displaying tasks of child workspaces

You can display all child workspaces' tasks in the task monitor of the parent workspace.

To do so, follow these steps:

  1. Go to the Administration page.

  2. Select the Show tasks of subworkspaces option.

As a result, all of the tasks from the child workspaces are displayed in the task monitor of the parent workspace.

Task details

Task status

The tasks are grouped into tabs based on the task's status:

  • All tasks - all tasks your workspace has access to

  • Failed - the tasks that resulted in an error

  • Warnings - the tasks that were completed with warnings

    Warnings may be raised by the following issues:

    • Issues that prevent your datastream from fetching data from your chosen data source, such as incorrect datastream configuration settings

    • Issues that occur while applying an transformation to your data extract, such as incorrect configuration of an transformation

    • Issues detected by data monitors, such as duplicate data rows in your data extracts. For more information, see Introduction to the Data Quality page.

  • Pending - the tasks that have not yet started

  • Running - the tasks that have started but not yet finished

  • Completed - the tasks that have finished successfully

Task actions

You can perform the following actions for each task in the task overview:

Restart

Click Restart to run the fetch task again. A restart deletes the data extract from the previous task and creates a new task at the top of the All Tasks panel with a new data extract.

Viewing data extracts

Click the task's overview to show the data extracts collected during the fetch.

Re-load

Click Re-load to resend the data to Adverity Data Storage or external destinations. The relevant data in the destination is overwritten. Use Re-load when you have made changes to Data Mapping or destination's configuration after a fetch.

Acknowledge

Each task with an error or a warning has an Acknowledge button. Acknowledged tasks are not displayed in the Failed or Warnings tabs.

For more information, see Acknowledging a task with issues.

Acknowledging a task with issues

You can acknowledge a task that has errors or warnings. Acknowledged tasks are not displayed in the Failed or Warnings tabs, but you can still see them in the All Issues tab.

Acknowledging a failed task does not affect the numbers in the New failed tasks (today) KPI box.

Tasks with issues are automatically acknowledged if a later fetch, which includes the whole date range of the task with issues, is completed successfully. For example, if a fetch for March 10-15 fails or has warnings, but a later fetch for the whole month of March is successful, then the first task will be acknowledged automatically.

Automatic issue acknowledgment is only supported for datastreams with the Unique by fetch local data retention configuration.

To acknowledge a task with errors or warnings, follow these steps:

  1. Go to the Activity page.

  2. (Optional) Click the Failed or Warnings tab to view all tasks with a specific issue type.

  3. Find a task you want to acknowledge.

  4. In the top right corner of the task box, click Acknowledge.

Task stages

The three progress bars in the task overview show the status of each stage of a task - fetching data, enriching data, and loading data into Adverity Data Storage and destinations.

General information

On the left, Adverity displays the connector used, and the time range for which data is collected.

Fetch

This stage displays the progress of a data fetch or the time it took to collect the data.

Transform

This stage displays the progress of data transformation or the time it took to transform the data in this task. If you have not assigned any transformations to this datastream, this stage is skipped.

Load

This stage displays the time it took to load the data in this task into Adverity Data Storage and/or external destinations. If you have not enabled Adverity Data Storage or assigned a destination to this datastream, this stage is skipped.

Errors & warnings

If a task stage fails, an error or warning message is displayed below the progress bar for that stage. Hover over the error or warning message for more information on the issue.

If a stage has failed, to run this stage again, click Retry.

An example of a successful task - this task involved manually fetching data from Google Sheets, enriching the data and loading data into Google Sheets

An example of a scheduled task that failed in the transformation stage - an error message is displayed under the failed stage, and the Acknowledge button appears in the top right corner

Skipped fetches

The task monitor displays a fetch task as Skipped if the fetch collects an empty data extract. A data extract is considered empty if it contains no columns and rows. The fetch task may collect an empty data extract for the following reasons, among others:

  • The data source contains no data for the date range specified in the fetch. For more information on collecting data, see Manual and scheduled fetches.

  • A transformation applied to the datastream filters out all data. For more information on transformations, see Transforming data in Adverity.