Available custom script instructions

The guides in this section of the Adverity documentation explain how to use the custom script instructions available in Adverity to tailor the data in your data extracts to meet your needs.

The instructions are sorted by the type of actions you can take using the instructions. Click the name of an instruction to find out how to configure this instruction as part of your custom script enrichment.

Add columns and rows to your data extract

Use these custom script instructions to add new columns or rows to your data extract. Some of these instructions add new fields with pre-determined or fixed values, and others allow you to use existing fields in the data extract to determine the values in your new columns and rows.

Instruction	Description
add_moving_average	Add a new column to the data extract with moving averages.
addfield	Add a new column to the data extract with a specific value in each row.
addfieldusingcontextx	Add a new column to the data extract with values calculated using data from the current, previous and next columns.
addfieldx	Add a new column to the data extract with values calculated using a Python expression.
addrownumbers	Add a new column to the data extract with row numbers.
append	Add rows to the data extract from another datastream or subtable.
capture	Add new columns to the data extract by capturing parts of the values from an existing column.
extendmissing	Check if certain columns exist in the data extract, and add them if they are missing from the data extract.
filltimegaps	Add rows to a data extract containing missing dates in a date range.
increment	Sort a data extract and add a new column containing a difference in values.
join	Join data extracts from multiple datastreams into a single data extract.
namingconvention	Add new columns to the data extract by using parts of the values from an existing column. This is a simplified version of the capture instruction.
resolveurl	Add a new column to the data extract containing a resolved URL.
splitfield	Split values into new columns using a regular expression.
sumup	Add a column containing a running total to your data extract.
unpack	Separate values contained in lists into columns in a data extract.
unpackdict	Separate values contained in dictionaries into columns in a data extract.
unpacklist	Separate values contained in lists into rows in a data extract.

Remove columns and rows from your data extract

Use these custom script instructions to select which columns or rows in your data extract to keep, and which to remove from your data extract.

Instruction	Description
cache	Cache the current state of the data table.
collect	Populate a value table using values from a certain column in a data extract.
conflicts	Find rows that share common values in a selected column but also contain non-identical values in at least one other column.
cut	Select and keep specific columns in a data extract.
cutout	Select and remove specific columns from a data extract.
cutoutre	Select and remove specific columns in a data extract using a regular expression.
cutre	Select and keep specific columns from a data extract using a regular expression.
delta	Compare data extracts, and return unique rows into a new data extract.
distinct	Find the number of distinct rows in a data extract.
duplicates	Find and return the duplicated rows in a data extract.
head	Return a set number of rows from the top of the data extract.
search	Search for certain values in a data extract and keep the rows containing the values.
select	Select and keep rows using a Python expression.
selecteq	Select and keep rows with a value that matches the value you enter.
selectfalse	Select and keep rows that are empty, or contain `null` or `0`, in a selected column.
selectgt	Select and keep rows with a number greater than the number you enter.
selectin	Select and keep rows with a value that matches any of a list of values you enter.
selectinvt	Select rows in a data extract to keep if they match values in a value table.
selectlt	Select and keep rows with a number less than the number you enter.
selectne	Select and remove rows with a value that matches the value you enter.
selectnone	Select and keep rows in a selected column that are empty.
selectnotin	Select and remove rows with a value that matches any of a list of values you enter.
selectnotnone	Select and remove rows in a selected column that are empty.
selecttrue	Select and keep rows that contain any value in a selected column.
skip	Remove a number of rows from the start of a data extract.
skipmeta	Download data directly from Adverity without a header in the file.
skipuntil	Remove rows from a data extract until a given value is found.
tail	Return a set number of rows from the bottom of a data extract.
trimheader	Remove leading or trailing whitespace from headers.
unique	Keep only the unique rows in a data extract.

Change the data in your data extract

Use these custom script instructions to transform the data in your data extract.

You might need to use one of these instructions to convert data (e.g. into numbers or Python datetime format) before using a different custom script instruction that requires data in this format.

Instruction	Description
aggregate	Convert existing fields in the data extract by performing calculations on selected values.
anonymise	Convert existing fields in the data extract by anonymizing the values in the selected fields.
applyschema	Rename columns in your data extract according to your Data Mapping.
convert	Convert the values in a column of a data extract.
convertall	Convert all the values in a data extract.
convertallx	Convert all the values in a data extract using a Python expression.
convertdates	Convert dates to Python datetime.
convertduration	Convert duration data into ISO 8601 format.
convertnumbers	Convert numerical characters to numbers.
convertpacked	Convert keys and values within a dictionary into a new format.
convertunix	Convert a UNIX timestamp into an easy-to-read datetime format.
convertx	Use a Python expression to convert values in a selected column.
currency	Convert monetary values from one currency to another.
fieldmap	Update your data extract table by combining existing values into new or existing columns.
filldown	Fill empty fields in the data extract using values in the row above.
fillright	Fill empty fields in the data extract using values in the column to the left of an empty column.
hash	Encrypt selected columns in a data extract.
melt	Reshape a table by transposing fields into data.
mergecolumn	Combine fields from different columns into a single column. The combined values are stored in a JSON-style dictionary.
mergeheader	Combine a defined number of rows into the table header.
mergesort	Combine data from a sorted data extract with another sorted data extract from a different datastream.
movefield	Move a column to a new position in the data extract.
preamble	Add new header rows and row values to the data extract.
pushheader	Add a new header to the data extract.
recast	Transpose distinct data values into the data extract header. This is the opposite of the melt instruction.
rename	Rename columns in a data extract.
reorder	Reorder columns in a data extract.
setheader	Rename the columns in your data extract.
sort	Sort data in a data extract.
timecast	Split a date range across multiple rows in a data extract.
translate	Translate values in your data extract from one language to another.
transpose	Swap the positions of rows and columns in a data extract.

Import data from a specific file type

Use these custom script instructions to import data from a file into your data extract.

Instruction	Description
avro	Import data from an Apache AVRO file.
csv	Parse a csv file.
excel	Import data from a specific sheet of an Excel file.
fixencoding	Decode or encode a data extract.
fwf	Import data with a fixed width from a text file.
json	Import data from a JSON file.
ndjson	Import data from a NDJSON (newline delimited JSON) formatted file.
parquet	Import data from a Parquet file.
recode	Detects the encoding of data and re-encodes to a given encoding type.
spss	Import data from an SPSS file.
text	Import data from a text file.
xlsx	Import data from an XLSX file.
xml	Import data from an XML file.
zipload	Import data from a ZIP file.

Manage your data extract

Use these custom script instructions to manage your data extract without changing the data it contains. You can use these instructions to perform a range of tasks, including the following:

Make changes to your data extract using metadata
Perform checks to verify the uniqueness and quality of your data
Export your data extract by email or in a range of file formats

Instruction	Description
dump	Save a copy of the data extract into a given file format.
expectheader	Check if the header contains the expected field names, and raise an error if issues are found.
if	Perform an action on a data extract if a condition is false.
mailto	Send a copy of a data extract by email.
map	Use a mapping table to create new values based on existing values in a data extract.
map_workspace	Send data to a workspace in Data Explorer & Dashboards.
metaaddfield	Add a column to the data extract containing selected data from the metaheader.
set_meta	Add values to the metadata of a data extract.
set_tags	Add custom tags to your data extract.
split	Split a data extract into multiple extracts.
verifyunique	Check if the rows in a data extract are unique.