Data types and measures

Data types and measures#

This guide provides information about the data types and measures used in Adverity.

Data types used in Adverity#

Adverity uses a number of data types for your data.

The table below provides information about the data types in Adverity that are universally used in data management.

Data type	Notes	Example
String	Use for dimensions, such as names, titles, and descriptors.	EN_Campaign_21
Long	Use for integer metrics without decimal points.	16
Float	Use for non-integer metrics with digits after a decimal point.	16.523
Date	Use for dates. Adverity recognizes common date formats. (Recommended) Use the ISO 8601 format `YYYY-MM-DD`.	2021-03-11
DateTime	Use for date and time. Adverity recognizes common datetime formats. (Recommended) Use the ISO 8601 format `YYYY-MM-DDThh:mm:ss`.	2021-03-11T16:23:41
Boolean	Use for binary metrics. The values are either 0 (false) or 1 (true).	0 or 1

The table below provides information about the data types that are specific to Adverity.

Data type	Notes	Example
Percentage	Use for rates and percentages. Adverity automatically multiplies the values by 100. For example, 0.1 is displayed as 10%.	0.1
Currency	Use for metrics that express values in a currency.	10
JSON	Use for layered metrics with JSON-compatible formatting.	{“firstName”: “Jane”, “lastName”: “Doe”}
Formula	Use to send information about formulas to the Google Sheets destination.	=A1 + B1
Duration	Use for metrics that express a time duration. (Recommended) Use the ISO 8601 format `PnYnMnDTnHnMnS`.	P2DT3H

Data type estimation in Adverity#

When you fetch data, Adverity automatically assigns a data type to each source field column in your data extract. When you configure Data Mapping for a column, you can choose a different data type for the target field assigned to this source field. The data type of the target field will be used when loading data into your destinations. For more information about configuring Data Mapping, see Applying Data Mapping to a datastream.

Limitations#

The Currency data type cannot be assigned based on estimation. This data type must be assigned to a field by configuring Data Mapping for the field or manually changing the source field type.

Data type weight hierarchy#

Adverity uses data type weights to select the best estimated type. The diagram below shows the data type weight hierarchy used during estimation, where String has the lowest weight and Boolean the highest. Data types on the same horizontal level have equal weights.

The default estimated data type is String.

Estimating data types in Adverity#

To estimate the data type for a column, Adverity uses the following steps:

Adverity determines all data types to which all values in the column can be converted.
Out of the possible data types, Adverity selects the type with the highest weight. This way the estimated type matches the column’s data the most closely.
If the estimated data type is Date, Adverity checks the format. If the date format contains “%H”, then the DateTime data type is assigned.

Examples of data type estimation#

Example 1

The source column contains JSON values.

The column’s values can be correctly converted to two data types: JSON and String. The JSON data type has higher weight than String. So Adverity estimates the column’s data type as JSON.

Example 2

The source column contains integer values.

The column’s values can be correctly converted to three data types: Long, Float and String. Out of these data types, Long has the highest weight. So Adverity estimates the column’s data type as Long.

Measures used for metrics in Adverity#

The table below provides information about measure types, the mathematical function underlying the values in a metric field.

Measure type	Notes
Sum	Use for metrics that display the sum of values. Applicable for most metrics with values that can be summed up across several data sets, such as clicks, impressions.
Average	Use for metrics that display the average of values by dividing the sum of values by the number of data sets.
Count	Use for metrics that display the number of data sets.
Min	Use for metrics that display the minimum value across all data sets.
Max	Use for metrics that display the maximum value across all data sets.
None (Metric)	Use for metrics that do not have an underlying mathematical function, such as campaign reach.
None (Dimension)	If you load data into Adverity Data Storage, Adverity treats the data type of a field that you set up with this measure as a String regardless of the data type you select in the Type field.

Date parsing in Adverity#

When you fetch data, Adverity automatically determines the format of date fields. This section explains the date parsing process and best practices for optimal performance.

How Adverity processes dates#

When you first fetch data in Adverity, the entire dataset is scanned to determine the appropriate data types for each column. During the following fetches, Adverity samples the data to verify format consistency.

To determine the date format, Adverity uses the following steps:

Adverity tries to parse dates using a fast method optimized for common date formats. For more information, see Pendulum date parser documentation.
If the fast method fails, Adverity switches to a more thorough but slower parsing method. For more information, see Dateutil date parser documentation.

This approach ensures both efficiency and compatibility with various date formats.

Standardizing date formats#

Adverity‘s convertDates custom script instruction helps ensure consistent date handling by converting date strings into a standardized datetime format. You can either let Adverity automatically detect the source date format or specify it manually using format patterns like %Y-%m-%d.

We strongly recommend using convertDates in your datastreams before performing any date-based operations to prevent processing errors and ensure reliable data analysis.

For more information, see ConvertDates.

Best practices for date handling#

To ensure optimal performance and accuracy in Adverity:

Use standard formats: The ISO 8601 format (YYYY-MM-DD for dates, YYYY-MM-DDThh:mm:ss for datetimes) provides the best parsing performance and reliability.
Maintain consistency: Use the convertDates custom script instruction to standardize date formats across your data sources. This helps prevent parsing errors and ensures consistent date handling throughout your datastreams.
Verify date formats: Always check that your source data’s date format matches your datastream configuration to avoid parsing errors and ensure accurate data processing.