duplicates

Find and return the duplicated rows in a data extract.

Introduction

Use the duplicates instruction to find and return the duplicated rows in a data extract. Unique rows are removed from the data extract.

Creating a custom script transformation using the duplicates instruction

To create and configure a custom script using the duplicates instruction, follow these steps:

1. Create a custom script transformation.

2. In the Instructions step, select the duplicates instruction.

3. To configure the custom script instruction, fill in the following fields. Required fields are marked with an asterisk (*).

Key

Enter the names of the columns where Adverity checks for duplicated rows.

Select a column in one of the following ways:

• Select String and enter the name of the column.

• Select Integer and enter the position of the column in the data extract as an index value. Counting starts at 0. To select the first column, enter 0. To select the second column, enter 1. To select the last column in the data extract, enter -1.

Leave this field empty to check all columns in the data extract.

Presorted

Select this checkbox if the rows in the Data Extract have been sorted before the transformation runs

Subtable

Enter the name for a subtable that you want to use within this custom script.

A subtable is a temporary table that only exists for this custom script. You can apply additional instructions within the same custom script to the subtable. However, the subtable cannot be used in any other custom scripts.

If a subtable does not exist for the current custom script, the transformation is applied to the data extract, and the enriched data is output into the subtable. If the subtable already exists for the custom script, the subtable is used as the input for the transformation and optionally as the output.

Example

Transformation configuration

Key*

Ad Group

Region

Data table before transformation

Ad Group	Region	Clicks
media	UK	3493
media	UK	8891
media	UK	2782
ecommerce	France	7397
ecommerce	Germany	6397
social	USA	5645
social	USA	1946
social	Spain	3356

Data table after transformation

Ad Group	Region	Clicks
media	UK	3493
media	UK	8891
media	UK	2782
social	USA	5645
social	USA	1946