The ChannelMix S3 connection utilizes the boto3 python library to connect to Amazon S3 buckets to either ingest data from files present in the bucket or to upload data to a requested bucket.
Connection Guide
Currently Supported Version: | |
---|---|
Attribution Window: | N/A |
Time Zone Reporting: | N/A |
API Limitations: | API limitations and quotas can be found here |
Backfill Limitations: |
N/A |
Refresh Limit: | N/A |
When selecting S3 in the ChannelMix connection library, a support ticket will automatically be generated and a ChannelMix support team member will assist you in setting up the connection. The support team member will create an S3 connection on your behalf and request that you fill in appropriate information depending on how you would like to grant ChannelMix permission to access the AWS S3 bucket.
AWS S3 Bucket Structure Recommendation:
Formatting the structure of your S3 bucket as shown below will help ChannelMix process your files more efficiently. Ultimately, you are the owner of the S3 bucket so the decision is yours alone and this is just a recommendation, but using this structure will help our Support team process your request more quickly.
s3://channelmix/<name_of_your_client>/<report_type>/file_name.csv
Example:
s3://channelmix/Client123/Campaign-Report/marketing_campaigns_sep_2024.csv
Things to avoid:
- Avoid using dates in folder names (it's ok to use dates in file names).
- Avoid partitioning files too granularly (unless file size requirements require you to do so).
- If today is 10/1/2024 and your report is meant to refresh the data for the month of September 2024, provide one file that contains the data for all days in September instead of providing one file per day in September.
- Ultimately, we want to be able to obtain as much data from one file as is reasonably possible. Usually, that means partitioning your file to include a month's worth of data.
Connection Authentication
ChannelMix supports these methods to connect to Amazon S3
- AWS Key and Secret
- Assume AWS Role (recommended)
Creating a ChannelMix Control Center Connection with AWS Key and Secret:
- Generate AWS Key and Secret using the information found below under "AWS Keys".
- Navigate to the CONNECT page in ChannelMix.
- Click on the "amazon S3" icon.
- Fill out the "AWS access key id" and "AWS secret access key" with the information generated in step 1.
- Enter "n/a" in the "Role ARN" field.
- Enter "n/a" in the "ChannelMix AWS Account ID" field.
Creating a ChannelMix Control Center Connection with Assume AWS Role:
- Follow the steps found below under "Assume Role".
- Navigate to the CONNECT page in ChannelMix.
- Click on the "amazon S3" icon.
- Use the Role ARN that you create in the "Creating a Role" section below to fill out the "Role ARN" field.
- The "ChannelMix AWS Account ID" should populate automatically.
AWS-Specific Instructions:
AWS Keys
To generate an access key and secret, please login to your AWS account and follow this guide provided by Amazon. Keep in mind that you are granting ChannelMix the same access as the user that is associated to the access key. User Guide article from Amazon.
Assume Role
Creating an AWS Role that ChannelMix can assume will allow access to be granted to ChannelMix and controled in one central location. The steps are a bit more involved but will provide the most flexibility and security as recommended by AWS.
Creating a Role
- In the AWS Console, navigate to Services > IAM
- Click "Roles" in the left navigation
- Click "Create Role"
- When asked to select a Trusted entity type choose "AWS account" and "Another AWS account"
- Account ID: 031871504755
- Additional Role Options: ChannelMix does not currently support requiring an external ID or MFA for assuming roles
- Add policies:
- AmazonS3FullAccess (if you would perfer to restrict ChannelMix access further, you can create a custom policy with the steps written below. Don't forget to add the policy to this Role when done!)
- Name the role
- Provide Role ARN to ChannelMix in the Connection. The ARN will look similar to this:
arn:aws:iam::011110000110:role/channelmix_s3_access
Creating a Custom Policy (optional)
If you do not want ChannelMix to use the built in AWS policy called "AmazonS3FullAccess" then you can create and manage a policy that has less access. Follow these steps:
- In the AWS Console, navigate to Services > IAM
- Click "Policies" in the left navigation
- Click "Create Policy"
- Select "S3" from Service selector
- The minimum access that ChannelMix requires is:
- s3:PutObject
- s3:GetObject
- s3:ListBucket
- s3:DeleteObject
- If you choose to restrict access to a specific resource (i.e. bucket) you can also add that information to the policy.
Below is a video walking through of the generic steps of creating a role and assigning policies. This video does
timestamps 1:28-2:50 are the walkthrough
Written documentation to go along with the video and further reading located here
Profile Association and Values
No configuration - n/a
User Permission Examples
{
"Version": "2024-05-16",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*",
"s3:Describe*",
"s3-object-lambda:Get*",
"s3-object-lambda:List*",
"s3:PutObject", // granular create/write can be broadened with s3:Put*
"s3:PutObjectTagging", // not needed if using s3:Put*
"s3:DeleteObject",// granular delete can be broadened with s3:Delete*
"s3:DeleteObjectTagging", // not needed if using s3:Delete*
"s3:AbortMultipartUpload:"
],
"Resource": [
"arn:aws:s3:::${BucketName}", // grants access only to the bucket
"arn:aws:s3:::${BucketName}/*" // grants access to all directories in bucket
]
}
]
}
A comprehensive list of all AWS Managed Policies can be found here, along with S3 specific actions or permissions, and finally documentation on S3 specific resources