Amazon S3 for Redshift

As discussed in cloud bucket storage setup, Amazon S3 is required for several data-related use cases in Sortment.

In AWS, accessing S3 via Redshift requires IAM Role ARN to prevent exposing user or access key details via SQL query.

For APIs, access is managed via Access Keys.

Connection configuration

Following details are requied on Sortment for S3 connection for Redshift:

• AWS Region: The geographical region where your AWS services, including S3 and Redshift, are deployed. Examples include us-east-1 (N. Virginia) or ap-south-1 (Mumbai). In your AWS Console → Click on your profile (top-right) → See the selected region.

• AWS S3 Bucket Name: A globally unique name assigned to an S3 bucket where you store and retrieve data. On AWS Console → Go to S3 → Locate your bucket under “Buckets” list.

• Access Key ID: A unique identifier for an AWS IAM user or service that allows programmatic access to AWS services. On AWS Console → IAM → Users → Select your user → Security Credentials tab → Generate Access Key.

• Secret Access Key: A secret key paired with the Access Key ID, used for signing API requests securely. Generated alongside the Access Key ID in IAM (only shown once). If lost, you must generate a new pair.

• IAM Role ARN: A unique identifier for an IAM Role that allows Redshift to interact with S3. On AWS Console → IAM → Roles → Find your role → Copy the Role ARN.

AWS credentials setup

This region for S3 Bucket should be same as your Redshift region for lower data egress costs.

Follow these steps to setup an S3 bucket for Sortment:

Creating a bucket

1. Sign in to the AWS Management Console and navigate to the S3 service.

2. Click on "Create bucket."

3. Enter a unique bucket name (globally unique across AWS).

4. Select an AWS region for the bucket. This should be same as your Redshift cluster region.

5. Create the bucket.

Creating a policy

1. From your AWS console navigate to Identity and Access Management (IAM) > Access Management > Policies

   
2. In the JSON policy editor, use the following policy. Update {$bucket-name} to the bucket created above. Give your policy a name and create the policy.

For Redshift to load data into an S3 bucket, it needs access to list all buckets it has access to. This is covered with resource level policy below.

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Action": "s3:ListBucket",
			"Resource": "arn:aws:s3:::{$bucket-name}"
		},
		{
			"Effect": "Allow",
			"Action": "s3:*",
			"Resource": "arn:aws:s3:::{$bucket-name}/*"
		}
	]
}

Creating a role

1. Now, navigate to Identity and Access Management (IAM) > Access Management > Roles and click Create Role.

   
2. Give access to the policy created for Sortment.

   
3. Next, setup the name and description and create the role.

4. Copy the role ARN. This will be used in S3 setup form on Sortment.

Associating role with your Redshift cluster

1. In Amazon Redshift console, choose Clusters, then choose the name of the cluster that you want to update.

2. In Actions dropdown, choose Manage IAM roles to display the current list IAM roles associated with the cluster.

3. On the Manage IAM roles page, choose the IAM role created earlier and then choose Add IAM role.

4. Save your changes to modify the cluster and allow Redshift access to the S3 bucket.

Configuring Access Key

The Access Key access type allows you to configure Sortment to use an IAM user by providing the user's Access Key ID and Secret Access Key.

If you need help generating these keys, consult the IAM article on this topic.

Once you have an Access Key ID and Secret Access Key, paste those values into the form and click Create.