Custom scripts available in Adverity

This reference documentation explains how to use custom script enrichments to tailor data to your needs.

The table below lists all available custom script enrichments.

Enrichment category

Enrichment name

Description

Add fields

add_moving_average

Add a new field to the data extract with moving averages.

Add fields

addfield

Add a new field to the data extract with a specific value in each row.

Add fields

addfieldusingcontextx

Add a new field to the data extract with values calculated using data from the current, previous and next rows.

Add fields

addfieldx

Add a new field to the data extract with values calculated using a Python expression.

Add fields

addrownumbers

Add a new field to the data extract with row numbers.

Convert fields

aggregate

Convert existing fields in the data extract by performing calculations on selected values.

Convert fields

anonymise

Convert existing fields in the data extract by anonymizing the values in the selected fields.

Add fields

append

Add fields to the data extract from another datastream or subtable.

Map/Order

applyschema

Rename fields according to Data Mapping.

Import data

avro

Load data from an Apache AVRO file.

Select/Cut fields

cache

Cache the current state of the data table.

Add fields

capture

Add new fields to the data extract by capturing parts of the values from an existing field.

Select/Cut fields

collect

Populate a value table using values from a certain column in a data extract.

Select/Cut fields

conflicts

Find rows that share common values in a selected column but also contain non-identical values in at least one other column.

Convert fields

convert

Convert the values in a column of a data extract.

Convert fields

convertall

Convert all the values in a data extract.

Convert fields

convertallx

Convert all the values in a data extract using a Python expression.

Convert fields

convertdates

Convert dates to Python datetime.

Convert fields

convertduration

Convert duration data into ISO 8601 format.

Convert fields

convertnumbers

Convert numerical characters to numbers.

Convert fields

convertpacked

Convert keys and values within a dictionary into a new format.

Convert fields

convertunix

Convert a UNIX timestamp into an easy-to-read datetime format.

Convert fields

convertx

Use a Python expression to convert values in a selected column.

Import data

csv

Parse a csv file.

Convert fields

currency

Convert monetary values from one currency to another.

Select/Cut fields

cut

Keep columns from a data extract.

Select/Cut fields

cutout

Remove columns from a data extract.

Select/Cut fields

cutoutre

Find columns to remove from a data extract using a regular expression.

Select/Cut fields

cutre

Find columns to keep in a data extract using a regular expression.

Select/Cut fields

delta

Compare data extracts, and return unique rows into a new data extract.

Select/Cut fields

distinct

Find the number of distinct rows in a data extract.

Process files

dump

Save a copy of the data extract into a given file format.

Select/Cut fields

duplicates

Find and return the duplicated rows in a data extract.

Import data

excel

Load data from a specific sheet of an Excel file.

Notifications

expectheader

Check if the header contains the expected field names, and raise an error if issues are found.

Add fields

extendmissing

Check if certain columns exist in the data extract, and add them if they are missing from the data extract.

Map/Order

fieldmap

Update your data extract table by mapping existing values into new or existing columns.

Map/Order

filldown

Fill empty fields in the data extract using values in the row above.

Map/Order

fillright

Fill empty fields in the data extract using values in the column to the left of an empty column.

Add fields

filltimegaps

Add rows to a data extract containing missing dates in a date range.

Import data

fixencoding

Decode or encode a data extract.

Import data

fwf

Load data with a fixed width from a text file.

Convert fields

hash

Encrypt selected columns in a data extract.

Select/Cut fields

head

Return a set number of rows from the top of the data extract.

Notifications

if

Perform an action on a data extract if a condition is false.

Add fields

increment

Sort a data extract and add a new column containing a difference in values.

Add fields

join

Join data extracts from multiple datastreams into a single data extract.

Import data

json

Load data from a JSON file.

Notifications

mailto

Send a copy of a data extract by email.

Map/Order

map

Use a mapping table to create new values based on existing values in a data extract.

Map/Order

map_workspace

Send data to a workspace in Data Explorer & Dashboards.

Convert fields

melt

Reshape a table by transposing fields into data.

Map/Order

mergecolumn

Combine fields from different columns into a single column. The combined values are stored in a JSON-style dictionary.

Map/Order

mergeheader

Combine a defined number of rows into the table header.

Map/Order

mergesort

Combine data from a sorted data extract with another sorted data extract from a different datastream.

Meta operations

metaaddfield

Add a column to the data extract containing selected data from the metaheader.

Map/Order

movefield

Move a column to a new position in the data extract.

Add fields

namingconvention

Add new fields to the data extract by using parts of the values from an existing field. This is a simplified version of the capture instruction.

Import data

ndjson

Load data from a NDJSON (newline delimited JSON) formatted file.

Import data

parquet

Load data from a Parquet file.

Map/Order

preamble

Add new header rows and row values to the data extract.

Map/Order

pushheader

Add a new header to the data extract.

Convert fields

recast

Transpose distinct data values into the data extract header. This is the opposite of the melt instruction.

Import data

recode

Detects the encoding of data and re-encodes to a given encoding type.

Map/Order

rename

Rename columns in a data extract.

Map/Order

reorder

Reorder columns in a data extract.

Add fields

resolveurl

Add a new column to the data extract containing a resolved URL.

Select/Cut fields

search

Search for certain values in a data extract and keep the rows containing the values.

Select/Cut fields

select

Select rows in data extract to keep using a python expression.

Select/Cut fields

selecteq

Select and keep rows in a data extract if they contain values that are an exact match to a given value. This search is not case sensitive.

Select/Cut fields

selectfalse

Select and keep rows in a data extract that contain empty fields in a chosen column.

Select/Cut fields

selectgt

Select rows in a data extract to keep if a value is greater than a given limit.

Select/Cut fields

selectin

Select and keep rows in a data extract if they contain values that are an exact match to a given value. This search is case sensitive.

Select/Cut fields

selectinvt

Select rows in a data extract to keep if they match values in a value table.

Select/Cut fields

selectlt

Select rows in a data extract to keep if a value is less than a given limit.

Select/Cut fields

selectne

Select rows in a data extract that contain a value that does not match a given value.

Select/Cut fields

selectnone

Select and keep rows in a data extract if they contain empty values in a certain column.

Select/Cut fields

selectnotin

Select rows in data extract to keep if they do not exactly match a given value.

Select/Cut fields

selectnotnone

Remove rows from a data extract if they contain empty values in a certain column.

Select/Cut fields

selecttrue

Select and keep rows in a data extract if they contain any value.

Meta operations

set_meta

Add values to the metadata of a data extract.

Meta operations

set_tags

Add custom tags to your data extract.

Map/Order

setheader

Rename the column in your data extract.

Select/Cut fields

skip

Remove a number of rows from the start of a data extract.

Select/Cut fields

skipmeta

Download data directly from Adverity without a header in the file.

Select/Cut fields

skipuntil

Remove rows from a data extract until a given value is found.

Map/Order

sort

Sort data in a data extract.

Process files

split

Split a data extract into multiple extracts.

Add fields

splitfield

Split values into new columns using a regular expression.

Import data

spss

Load data from an SPSS file.

Add fields

sumup

Add a running total to your data extract.

Select/Cut fields

tail

Return a set number of rows from the bottom of a data extract.

Import data

text

Load data from a text file.

Convert fields

timecast

Split a date range across multiple rows in a data extract.

Convert fields

translate

Translate values in your data extract from one language to another.

Map/Order

transpose

Swap the positions of rows and columns in a data extract.

Select/Cut fields

trimheader

Remove leading or trailing whitespace from headers.

Check values

unique

Keep only the unique rows in a data extract.

Add fields

unpack

Separate values contained in lists into columns in a data extract.

Add fields

unpackdict

Separate values contained in dictionaries into columns in a data extract.

Add fields

unpacklist

Separate values contained in lists into rows in a data extract.

Check values

verifyunique

Check if the rows in a data extract are unique.

Import data

xlsx

Load data from an XLSX file.

Import data

xml

Load data from an XML file.

Import data

zipload

Load data from a ZIP file.