Collecting data from Box
This guide explains how to collect data from Box. To learn how to collect data from a different data source, go back to the Available data sources in Adverity overview.
Creating a datastream to collect data from Box
The basics of creating a datastream to collect data from any data source are explained in our guide to Creating a datastream. This guide contains information about the specific steps to create a datastream to fetch data from Box.
Configuration: Choose the data you want to collect from Box
The Box connector has many optional fields to configure. This section covers only the mandatory fields to configure for the connector. For a complete description of the optional fields to configure, see
To choose what data to collect and customize the Box datastream configuration, follow these steps:
-
(Optional) Rename your datastream.
-
In Source URL, enter the sub-folder where the data files are found.
-
Click Test to check the .
-
In File pattern, enter the name of the file to collect data from. To collect data from more than one file, specify a regular expression that matches the filenames. For more information on how to configure the File pattern field, see Advanced Box tips.
For information on configuring other Box fields, see Advanced Box tips.
What's next?
-
Apply Data Mapping to your collected data to harmonize data collected from different sources in Adverity.
-
Transform your data to meet your needs by creating and applying enrichments to your datastream.
-
Load your data into Explore & Present to visualize your data in Adverity
-
Load your data into an external destination of your choice
Advanced Box tips
Configuring data collection from a file server
In the Settings tab of your datastream overview, you can configure a number of additional settings:
-
Warnings enabled
-
Select this checkbox to display a warning in the datastream Overview page when no file is found on the file server or the file server URL is invalid.
The File Parsing section contains the following fields:
-
Parse
-
Select the data format used in the source files. The type of data format selected may cause additional fields to appear.
-
Delimiter
-
If CSV is selected in Parse, specify the character used to separate values in the CSV source files.
-
Quote Char
-
If CSV is selected in Parse, specify the character used to quote values with special characters in the CSV source files.
-
Quoting
-
If CSV is selected in Parse, specify when a quoting character should be added to the field values in the data extract. Choose from one of the following options:
-
Select all to add quote characters to everything in the data extract, regardless of the field type.
-
(Default) Select minimal to add quote characters only when required. For example, a quote character will be added to a field that contains either the Quote Char or Delimiter.
-
Select none to ensure no quote characters are added to the data extract.
-
Select nonnumeric to add quote characters to everything, except integer and float values.
-
-
Sheet
-
If Excel is selected in Parse, specify the name of the sheet within the Excel source files to import to Adverity.
-
Column offset
-
If Excel is selected in Parse, specify the number of columns that you do not want to import to Adverity from each Excel source file. For example, if the first column contains information that you do not want to import, specify 1 in this field.
-
Row offset
-
Specify the number of rows that you do not want to import to Adverity from each source file. For example, if the first row contains header information that you do not want to import, specify 1 in this field. This field is available for all parse types.
-
Skip initial space
-
If CSV is selected in Parse, select this checkbox to ignore any whitespace that follows the selected delimiter character.
-
Fix broken row counts
-
If Excel is selected in Parse, select this checkbox to fetch all rows from the data source if you notice a difference in the row counts.
The File Matching Options section contains the following fields:
-
File pattern
-
Specify a regular expression that matches the file attachment from which to collect data. By default, File pattern is pre-populated with
.*
which collects data for any attached file. -
Other examples of regular expressions for file patterns include:
-
.*\.xlsx
collects every file that ends with .xlsx. -
.*\.csv
collects every file that ends with .csv. -
.*Display.*\.csv
collects every file that contains the (case-sensitive) phrase Display anywhere in the file name and also ends with .csv.
-
-
Source encoding
-
Select the character encoding used in the source files. By default, the option Auto-detect is selected and will automatically detect the encoding of the source files.
-
Zip match
-
If you collect data from ZIP or GZIP files, specify a regular expression that matches the file names within the ZIP or GZIP containers. Leave this field empty to collect all files within the ZIP or GZIP container.
-
Archive password
-
If the ZIP or GZIP files require password access, provide the password. By default, Adverity uses the previously entered password, if available.
-
Keep filename
-
Select this checkbox to name data extracts using the same name as the corresponding source files. If you select this checkbox and the source filename stays the same between fetches, data in the corresponding data extract is overwritten.
The File Processing section contains the following fields:
-
Process all
-
By default, Adverity only processes the most recently uploaded files. Select this checkbox to process all files that match the criteria you specify.
-
Recursive
-
By default, Adverity only searches for source files in the folder that you specify. Select this checkbox to search in all the subfolders of the specified folder.
-
Concatenate files
-
Select this checkbox to combine data from all source files into a single data extract.
-
Delete source
-
Select this checkbox to delete the source files after Adverity has imported their data.
-
Move to
-
Specify the full path to the folder in into which the source files will be moved after Adverity has imported the data.
Uploading files into Adverity
To upload a file into Adverity using the Box datastream, follow these steps:
-
Go to the Datastreams page.
-
In the top right corner of the page, click Upload.
-
Click Choose File, and select the file to upload.
-
(Optional) Select the Keep data in raw state checkbox to achieve the following goals:
-
Keep the data in its original form and do not apply any enrichments assigned to this datastream. For more information on enriching your data, see Enriching data in Adverity.
-
Fetch the data without loading it into the destination specified for this datastream. For more information on loading data into a destination, see Loading data into destinations.
-
-
Click Upload.
As a result, the file is uploaded into Adverity and Adverity creates a data extract that includes the uploaded data.
Fetching files from Box manually
To browse files in the Box server and fetch them manually, follow these steps:
-
Go to the Datastreams page.
-
In the top right corner of the page, click More .
-
Click Browser.
-
Select the files to fetch. If the file is in a folder, click on the folder name to open the folder and view the files.
-
Click Fetch selected files.
The fetch collects data from Box which takes some time. The Overview page of the newly created datastream is now displayed. To preview the collected data, follow these steps:
-
In the All tasks tab, find the task at the top of the list, and click Show extracts.
-
Click the top hyperlink.
-
The data extract is displayed in a table containing the data that you have fetched.