Troubleshooting errors when collecting and loading data

Troubleshooting errors when collecting and loading data#

This guide explains how to troubleshoot errors that may happen when collecting and loading data.

Introduction#

Error messages for data collection and loading are displayed in the Failed tab of the task monitor. For more information, see Using the Activity page.

Fetch errors#

Errors marked as Fetch Error come from the data source from which you collect data. Adverity does not handle these error messages individually.

To resolve fetch errors, try the following:

  • Ensure the datastream is properly configured.

    If the data source does not support the combination of fields you selected in the datastream configuration, change the list of selected fields.

  • Ensure the authorization to the data source functions properly.

    If an authorization’s authorization to connect to the data source expires, update the authorization. For more information, see Updating an authorization.

    If an account with which you authorized authorizations no longer has access to the data source, delete the affected authorizations.

  • The data source API or file server is unavailable and Adverity cannot reach it to collect data.

    If the time of the last successful fetch is more recent than the error message, the data has already been fetched successfully since the failure. Click Acknowledge to acknowledge the error.

    If Adverity could not perform a successful fetch since the error message, click image1 Retry to try fetching the data again.

Fetch error No space left on device#

If Adverity runs on your own server, not in a cloud server, you might see the error [Errno 28] No space left on device. The reason can be the following:

  • You have too many data extracts in Adverity.

  • You performed multiple large fetches at the same time.

To resolve the error, try the following:

  • Delete unused datastreams.

  • Change the Retention type option in the advanced settings of your datastreams. For more information, see Configuring advanced datastream settings. For example, select Retain N fetches and specify 40 in the Retention Number field. As a result, data from this datastream is only retained for the last 40 fetches. Data from older fetches is deleted the next time you collect data with this datastream.

Fetch status stays Collected#

If the status of a fetch stays Collected and data is not loaded into the destination, try the following:

  • Ensure the datastream status is live.

  • Ensure a destination is selected and enabled for the datastream.

  • If you load data into AWS S3, ensure that the AWS policy file is correctly configured. For more information, see Configuring AWS policies.

Load error involving key columns#

Key columns are used to uniquely identify rows in a data table. If the combination of dimensions you set as key columns do not uniquely identify rows in the data extract that you want to load into a destination, you may receive an error message similar to the following:

  • Duplicate key value violates unique constraint

  • Could not create unique index

To resolve the error, try the following:

  • Change the key columns so that they uniquely identify rows in the data extract. For more information, see Setting key columns.

  • Aggregate metrics for rows that the existing key columns do not uniquely identify.

Warning

After changing the key columns, you need to perform additional actions to load data into some destinations. For an example, see Advanced SQL Database tips.

To overwrite data in a destination based on key columns, select Key Columns in the destination configuration. For more information, see Configuring data loading settings.

Some destinations do not support overwriting data with key columns. For an example, see Advanced Google BigQuery tips.

Contents