Using placeholders to name objects

This article explains how to use placeholders to name files and tables.

Introduction

Use placeholders to create dynamic names for objects such as data tables or files when loading a data extract into Adverity Data Storage or a destination. Placeholders use the metadata from a data extract. Only certain fields allow the use of placeholders, for example, they can be used to name a table created in a database destination when loading a data extract into Adverity Data Storage or a destination.

Placeholders do not rename existing files in Adverity, instead, they are used to create names for objects found in external destinations.

Use unique, dynamic names to prevent data from being overwritten in the destination. Using static or hard-coded names may result in data being overwritten.

Viewing available metadata

The {meta[*]} placeholder requires metadata from a data extract. To view the list of metadata available for use in this placeholder, follow these steps:

  1. Go to the Activity page.

  2. In the middle of the page, in the All tasks tab, find the task with the metadata you want to view, and click Show extracts.

  3. In the list, find the data extract with the metadata you want to view, and click the hyperlinked element.

  4. At the top of the page, click Metadata.

Adverity displays a list of metadata specific to that data extract in the middle of the page. The values in the left column can be used within the {meta[*]} placeholder.

The list of metadata values varies slightly depending on the type of datastream from which the data has been fetched. However, many metadata values are common to all data extracts. A list of these common metadata values is shown in the Table of common metadata values.

Creating names with placeholders

Use placeholders in an applicable field to create dynamic names for objects such as:

  • File names when loading data into destination types of files or datalakes.

  • Table names when loading data into external databases.

Static or hard-coded names may result in data from being overwritten. Dynamic names prevent data being overwritten due to the creation of a unique name.

See the example below for how to create a dynamic name with the {scheduled_year} and {extract_id} placeholders.

  1. Go to the Destinations page.

  2. Click the destination in which you want to create a dynamic name.

  3. Find the relevant field that allows placeholders. Such fields can be found in the following locations:

    • For file or datalake destinations (such as File OneDrive), click File Format, the File Name Template field allows the use of placeholders.

    • For database destinations (such as Azure Synapse), under Configuration, the Table Name Template field allows the use of placeholders.

    • For most destinations, in Datastream Mappings, the fields within the Table name or File name column allow the use of placeholders.

  4. In the relevant field, combine the placeholders {scheduled_year} and {extract_id} together as {scheduled_year}_{extract_id}.

This creates a dynamic name similar to 2021_12345. In this example, 2021 is the year on which the data extract took place and 12345 is the data extract ID.

Creating names with the meta placeholder

The placeholder {meta[*]} uses metadata from a data extract, where the * represents an item of metadata. When using the {meta[*]} placeholder, replace * with an item of metadata found in a data extract.

To find an item of metadata, follow the steps outlined in Viewing available metadata.

See the example below for how to use the {meta[*]} placeholder with the metadata value datastream display name to create a dynamic name.

  1. Go to the Destinations page.

  2. Click the destination in which you want to create a dynamic name.

  3. In the relevant name field that allows placeholders, enter the placeholder {meta[*]}.

  4. Replace * with datastream_display_name, so the placeholder becomes {meta[datastream_display_name]}.

The name is created using the datastream_display_name from the metadata of the data extract.

For example, the metadata item datastream_display_name for a Facebook Ads Insight data extract is Facebook Ads Insight. Therefore, the placeholder of {meta[datastream_display_name]}_{extract_id} would create the name Facebook Ads Insight_12345.

Table of common metadata values

The metadata of a data extract contains a series of information unique to that data extract. This metadata is available to use within the {meta[*]} placeholder.

The list does not contain all possible metadata values, as the list varies depending on the type of datastream. Instead, this table lists the most common metadata values.

Metadata value

Description

Example

adverity_datatap_version

Version number of Adverity at the time of the data extract.

2021.12.0.3

custom_meta_information

Additional, source specific information, most commonly used in file-based connectors.

{'source_directory': None, 'source_encoding': 'utf-8', 'source_filename': 'Basic ScriptingTable.xlsx.gz'}

datastream_URI

Uniform Resource Identifier (URI) of the datastream

anon.datatap.adverity.com/ads_insights/123/facebook-ads-10/

datastream_callback_URL

URL of the datastream. This is a hyperlink to the datastream page.

https://anon.datatap.adverity.com/ads_insights/123/facebook-ads-10/

datastream_display_name

Name of the datastream.

Facebook Ads 10

datastream_extract_URI

Uniform Resource Identifier (URI) of the data extract preview file.

anon.datatap.adverity.com/ads_insights/123/facebook-ads-10/ads_insights-123-20210301-a724a297e86dfd5d2db077ea35201723.csv/

datastream_extract_callback_URL

URL of the data extract preview file. This is a hyperlink to the data extract preview.

https://anon.datatap.adverity.com/ads_insights/123/facebook-ads-10/ads_insights-123-20210301-a724a297e86dfd5d2db077ea35201723.csv/

datastream_extract_column_count

Number of columns within the data extract.

50

datastream_extract_created

Date and time on which the data extract took place.

2021-03-01 09:48:53

datastream_extract_datepattern

Format of the date.

yyyy-MM-dd

datastream_extract_display_name

Display name of the data extract.

Facebook Ads 10 - 2021-03-01

datastream_extract_id

ID of the data extract.

2886

datastream_extract_is_empty

True/False flag to indicate if a data extract is empty.

False

datastream_extract_mapping

Overview of the datastream mapping applied to the data extract.

{'account_id': 'column': 'advertiser_id', 'type': 'string'}

datastream_extract_range_end

End date of the date range of the data extract.

2021-02-28 23:59:59

datastream_extract_range_start

Start date of the date range of the data extract.

2021-02-01 00:00:00

datastream_extract_row_count

Number of rows within the data extract.

560

datastream_extract_scheduled

Date and time on which a data extract was scheduled to run.

2021-03-01 08:00:00

datastream_extract_updated

Date and time on which a data extract was updated.

2021-03-01 09:48:53

datastream_id

ID of the datastream.

4321

datastream_overwrite_options

List of clauses that determine if a data extract should overwrite existing data within the destination.

{'dimensions': [], 'on_duplicate_filename': False, 'on_matched_date_range': True, 'same_data_stream': True}

datastream_type_display_name

Display name of the datastream.

Facebook Ads Insights

datastream_use_datatap_endpoint

True/False flag to indicate if the data extract destination is within Adverity.

True

datatap_job_uuid

Universally unique identifier (UUID) of the datastream task associated with the data extract.

49e872cc-1912-4262-9192-e5d7000b761d

extract_date

Date on which the data extract took place.

2021-03-01

managed_name

True/False flag to indicate if Unique by day has been selected in the Extract Filenames field. This option is found in the Local Data Retention section of the datastream configuration. For more information, see Configuring advanced datastream settings.

True