Using the Performance Manager

This reference article explains the different sections of the performance manager page.

Introduction

The Performance Manager page displays an overview of how long it took a scheduled task to complete, also known as a task duration. This page also displays performance outliers as well as recommendations and alerts. Performance outliers are Datastreams with tasks that took longer than expected to complete. Recommendations and alerts are messages regarding Datastream performance issues.

The Performance Manager page only displays information for the Datastreams within your chosen Workspace.

Recommendations and Alerts

The recommendations and alerts are displayed at the top of the Performance Manager page. This section of the page displays messages regarding Datastream performance issues.

An example of a performance issue message is as follows: 2 Datastreams are blocking your Instances performance. Learn More.

This alert means there are Datastreams in your Workspace that are consuming a lot of resources resulting in other tasks becoming blocked due to a lack of resources. To find out which Datastreams are consuming the resources, click Learn more. This opens a Performance manager details overlay that lists the Datastreams consuming the resources.

We recommend scheduling tasks evenly throughout the night. Adverity automatically attempts to evenly distribute scheduling times between 00:00 and 06:00.

Performance Outliers

The performance outliers section of the page lists the Datastreams with tasks that took longer than expected to complete. The table of outliers lists the following information:

Datastream Name

The name of the Datastream with the outlier task.

Increase

The percentage increase in task duration compared to the benchmark completion time.

Performance

The time the outlier task took to complete, displayed in HH:MM:SS format.

Benchmark

The expected task completion time. The performance is measured against the benchmark to calculate the increase. For example, if a task has a benchmark completion time of an hour but instead takes two hours to complete, the calculated increase is 100%. The benchmark time is calculated using a rolling average of completion times for previously completed tasks.

Description

The description informs you which part of the task process caused the outlier to occur. Below are typical examples of outlier descriptions.

  • Fetch time increased by 00h:06m:26s

  • Total duration 00h:14m:52s more than usual

  • 14 more API Retries (This message indicates an error in the Datastream.)

The performance manager page displays information on performance outliers that occurred in the last seven days. By default, only the most recent outliers are displayed in the table, with one outlier per Datastream shown. To view all the performance outliers that have occurred in the last seven days, click See {Number} More (where {Number} is the number of additional outliers to view).

Solving outlier issues

Below is a list of solutions for tasks that are taking too long:

  • To reduce the Fetch phase, fetch smaller amount of data.

  • To reduce the Enrich phase, reduce the complexity of the assigned enrichments.

  • To reduce the Transfer phase, review the configuration of your assigned Destinations.

  • Long idle or waiting times between phases may be caused by too many tasks scheduled to take place at the same time.

Task durations

The task durations are displayed in a task table that shows tasks from every Datastream within your Adverity Workspace. The most recently completed task is displayed at the top. The table lists the time window in which the scheduled task started, the name of the Datastream with which the task is associated, and how long the task took to complete. An example of the task table is shown in the figure below.

The task duration table within the Performance Manager page. An example of a tool tip is also displayed.

Understanding the task duration bar

The task duration is displayed as a colored bar, where each color represents a different task status. A task duration bar may contain more than one color. For example, a task duration bar may contain a brown section and then a cyan section, indicating the task was delayed before successfully completing.

These color representations are as follows:

  • Red indicates there was an error while processing the task.

  • Yellow indicates a task has been canceled.

  • Brown indicates a start delay for the scheduled task.

  • Cyan indicates a task was completed successfully.

  • Gray indicates a task is still in progress.

  • Blue indicates a task is pending.

Hover over a task duration bar to view a tool tip. These tool tips display the following:

  • How long the task took to process.

  • The number of enrichments applied.

  • The number of Destinations to which the data is being transferred.

  • The cause of a task being idle.

If a task has not been completed, the tool tip will display an applicable message. Examples of these tool tips are shown in the figure above.

Filtering tasks in the Performance Manager page

To filter which tasks are displayed in the Performance Manager page, follow these steps:

  1. Click the Connect element and select the Workspace you work with in Connect, Enrich & Transfer.

  1. In the left navigation panel, click Performance Manger.

  2. In the Performance Manager page, apply the following filters:

    Datasources

    Filter the task duration table to display only tasks associated with Datastreams collecting data from a certain data source.

    Filter by Task duration

    Apply this filter to display tasks that were completed within a certain time frame.

    Filter by task status

    Apply this filter to display tasks that have a certain status.

    Scheduled

    Choose to display tasks scheduled to start in the Last 12 hours or the Last 24 hours. By default, Adverity displays tasks scheduled to start in the Last 12 hours.