Advanced AWS policy configuration

Advanced AWS policy configuration#

This guide explains how to configure AWS policies to allow Adverity to access your Amazon S3 resources.

Introduction#

To allow Adverity to access your Amazon S3 buckets, you need to configure AWS policies in your AWS account. There are two types of policies you may need:

Trust policy: Required if you use role assumption to authorize Adverity. A trust policy grants Adverity permission to assume a role in your AWS account.
Permissions policy: Defines what actions Adverity can perform on your S3 resources (such as reading, writing, or listing objects).

This guide explains:

How to configure a trust policy for role assumption
What permissions policies are required for using S3 as a destination, data source, or storage

Prerequisites#

Before you read this reference, perform all of the following actions:

Ensure you can access IAM console.
Ensure you can access the AWS Policy Generator.
Ensure you know the Bucketname and Pathname of your AWS S3 bucket. For more information on S3 Bucket properties, see the AWS documentation.

Configuring a trust policy for role assumption#

If you want Adverity to access your S3 bucket using AWS role assumption instead of access keys, you need to configure a trust policy in AWS that allows Adverity to assume your role.

Important

Role assumption relies on Adverity assuming your role from Adverity’s own AWS account, so this approach is only available if your Adverity instance is hosted by Adverity. If your instance is deployed in your own cloud environment or installed on-premises, use Access keys instead. For more information, see the authorization steps in Amazon S3: Setup guide.

Note

Role assumption is an alternative to using Access Key ID and Secret Access Key. With role assumption, you create an IAM role in your AWS account and grant Adverity permission to assume that role.

Understanding role assumption#

When using role assumption:

You create an IAM role in your AWS account with the necessary S3 permissions.
You configure a trust policy on that role to allow Adverity to assume it.
In Adverity, you enter your own role ARN in the AWS Role ARN to assume field.

Adverity ARN for trust policies#

The ARNs in this section identify Adverity’s own AWS accounts. They apply only if your Adverity instance is hosted by Adverity — do not add them to your trust policy if your instance is deployed in your own cloud environment or installed on-premises.

To allow Adverity to assume your role, add your instance’s IAM Role ARN to your role’s trust policy in AWS.

Tip

To get the exact ARN for your instance, view the IAM Role field on the System Environment page. You need Administrator permissions to access this page. For more information, see Viewing system environment details.

Below is an example trust policy that allows Adverity to assume your role. Replace the ARN with your instance’s IAM Role ARN:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ACCOUNT_ID:role/datatap-instance"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Using External ID for additional security#

For additional security, you can use an External ID to prevent the “confused deputy problem”. When you specify an External ID:

Add a condition to your trust policy that requires the External ID.
Enter the same External ID in the External ID field in Adverity when setting up your Amazon S3 authorization.

Below is an example trust policy with an External ID condition. Replace the ARN with your instance’s IAM Role ARN:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ACCOUNT_ID:role/datatap-instance"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": "YOUR_EXTERNAL_ID"
                }
            }
        }
    ]
}

Replace ACCOUNT_ID with the account ID from your instance’s IAM Role ARN, and YOUR_EXTERNAL_ID with a unique identifier of your choice.

For more information on the confused deputy problem and External IDs, see the AWS documentation.

Configuring permissions policies#

The following sections explain what permissions policies are required depending on how you use S3 with Adverity.

Using S3 as a destination#

To use an S3 Bucket as a destination, ensure the AWS Policy contains the following statements:

Action

To use an S3 Bucket as a destination, you must include the following actions in the actions element of the AWS policy:

s3:DeleteObject
s3:GetObject
s3:PutObject
s3:ListBucket - Include this action in a separate statement of the AWS policy that also contains a condition. See the example AWS policy below.

Resource

Provide the pathway to the S3 Bucket in the following format:

arn:aws:s3:::BUCKETNAME/PATHNAME/*

For the AWS Policy statement with the action s3:ListBucket, include the following resource element:

arn:aws:s3:::BUCKETNAME

For more information on the resource element of the AWS policy, see the AWS documentation.

Condition

Configure the condition element to control when the policy takes effect. Include the condition element in the AWS policy statement that includes the action s3:ListBucket. See the example AWS policy below.

Use the condition operator StringLike with the condition key s3:prefix to make sure the AWS policy only works for a certain resource pathname. An example of the condition element is as follows:

"Condition": {
    "StringLike": {
        "s3:prefix": "PATHNAME*"
        }
}

For more information on the condition element of the AWS policy, see the AWS documentation.

Example of an AWS permissions policy when using S3 as a destination#

Below is an example of an AWS permissions policy to use an S3 bucket as a destination.

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Action": [
               "s3:DeleteObject",
               "s3:GetObject",
               "s3:PutObject"
           ],
           "Resource": [
               "arn:aws:s3:::BUCKETNAME/PATHNAME/*"
           ],
           "Effect": "Allow",
           "Sid": "Stmt1478773756000"
       },
       {
           "Action": [
               "s3:ListBucket"
           ],
           "Resource": [
               "arn:aws:s3:::BUCKETNAME"
           ],
           "Effect": "Allow",
           "Sid": "Stmt1478773807000",
           "Condition": {
             "StringLike": {
               "s3:prefix": "PATHNAME*"
              }
           }
       }
   ]
}

Using S3 as a data source#

To use an S3 Bucket as a data source, ensure the AWS Policy contains the following statements:

Action

To use an S3 Bucket as a data source, you must include the following actions in the actions element of the AWS policy:

s3:GetObject
s3:ListBucket - Include this action in a separate statement of the AWS policy that also contains a condition. See the example AWS policy below.

Resource

Provide the pathway to the S3 Bucket in the following format:

arn:aws:s3:::BUCKETNAME/PATHNAME/*

For the AWS Policy statement with the action s3:ListBucket, include the following resource element:

arn:aws:s3:::BUCKETNAME

For more information on the resource element of the AWS policy, see the AWS documentation.

Condition

Configure the condition element to control when the policy takes effect. Include the condition element in the AWS policy statement that includes the action s3:ListBucket. See the example AWS policy below.

Use the condition operator StringLike with the condition key s3:prefix to make sure the AWS policy only works for a certain resource pathname. An example of the condition element is as follows:

"Condition": {
"StringLike": {
    "s3:prefix": "PATHNAME*"
    }
}

For more information on the condition element of the AWS policy, see the AWS documentation.

Example of an AWS permissions policy when using S3 as a data source#

Below is an example of an AWS permissions policy to use an S3 bucket as a data source.

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Action": [
               "s3:GetObject"
           ],
           "Resource": [
               "arn:aws:s3:::BUCKETNAME/PATHNAME/*"
           ],
           "Effect": "Allow",
           "Sid": "Stmt1478773756000"
       },
       {
           "Action": [
               "s3:ListBucket"
           ],
           "Resource": [
               "arn:aws:s3:::BUCKETNAME"
           ],
           "Effect": "Allow",
           "Sid": "Stmt1478773807000",
           "Condition": {
             "StringLike": {
               "s3:prefix": "PATHNAME*"
              }
           }
       }
   ]
}

Using S3 as storage#

To use an S3 Bucket as storage, ensure the AWS Policy contains the following statements:

Action

To use an S3 Bucket as storage, you must include the following actions in the actions element of the AWS policy:

s3:DeleteObject
s3:GetObject
s3:PutObject
s3:ListBucket - Include this action in a separate statement of the

AWS policy that also contains a condition. See the example AWS policy below.

Resource

Provide the pathway to the S3 Bucket in the following format:

arn:aws:s3:::BUCKETNAME/PATHNAME/*

For the AWS Policy statement with the action s3:ListBucket, include the following resource element:

arn:aws:s3:::BUCKETNAME

For more information on the resource element of the AWS policy, see the AWS documentation.

Example of an AWS permissions policy when using S3 as storage#

Below is an example of an AWS permissions policy to use an S3 bucket as storage.

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Action": [
               "s3:DeleteObject",
               "s3:GetObject",
               "s3:PutObject"
           ],
           "Resource": [
               "arn:aws:s3:::BUCKETNAME/PATHNAME/*"
           ],
           "Effect": "Allow",
           "Sid": "Stmt1478773756000"
       },
       {
           "Action": [
               "s3:ListBucket"
           ],
           "Resource": [
               "arn:aws:s3:::BUCKETNAME"
           ],
           "Effect": "Allow",
           "Sid": "Stmt1478773807000",
       }
   ]
}

Additional permissions for Amazon Redshift bulk loading#

If you use this S3 bucket as the Storage for an Amazon Redshift destination, Amazon Redshift reads the staged file itself as part of the COPY command, using the same credentials Adverity uses to write it.

Note

s3:GetObject is already required for S3 storage and is included in the example policy above, but it is worth confirming: this is the action that lets Amazon Redshift, not only Adverity, read the staged file. If it is missing, the first load into the destination fails.

Beyond that, the AWS Policy needs one additional action:

s3:GetBucketLocation - Required in addition to s3:ListBucket. Include it in the same AWS Policy statement as s3:ListBucket, using the bucket resource element arn:aws:s3:::BUCKETNAME.

If the bucket uses server-side encryption with a customer-managed AWS KMS key (SSE-KMS), the same principal also needs the kms:Decrypt, kms:GenerateDataKey, and kms:Encrypt actions on that key, and the key’s own key policy must allow them. Buckets using SSE-S3 (AWS-managed keys) need no additional configuration.

Applying AWS policies#

Trust policies and permissions policies are configured in the AWS IAM console:

Trust policies are attached to IAM roles. To create or modify a trust policy, edit the role’s trust relationships in the IAM console.
Permissions policies are attached to IAM users or roles. For more information on how to attach and test a permissions policy, see the AWS documentation.

Advanced AWS policy configuration

Contents

Advanced AWS policy configuration#

Introduction#

Prerequisites#

Configuring a trust policy for role assumption#

Understanding role assumption#

Adverity ARN for trust policies#

Using External ID for additional security#

Configuring permissions policies#

Using S3 as a destination#

Example of an AWS permissions policy when using S3 as a destination#

Using S3 as a data source#

Example of an AWS permissions policy when using S3 as a data source#

Using S3 as storage#

Example of an AWS permissions policy when using S3 as storage#

Additional permissions for Amazon Redshift bulk loading#

Applying AWS policies#