Data & Processing | Can ChannelMix Pull Parquet Files From S3?

Yes, ChannelMix can pull .parquet files from S3.  Follow the steps below to get this set up successfully.

1. S3 Connection:
   1. If the S3 bucket is not owned by ChannelMix, create an S3 connection in the ChannelMix Control Center.
   2. If the S3 bucket is owned by ChannelMix and already created, please make note of the file path.
   3. If you want to place .parquet files in an S3 bucket owned by ChannelMix that isn't yet created, please let us know within the ticket that is sent in step 3 to Platform Support.
2. Loading .parquet files to S3:
   1. The data to be loaded for the current day should have the day's date formatted as YYYYMMDD as the name of the folder (not the file).
      1. Example: s3://examplebucketname/foldername/YYYYMMDD/anotherfolder/filename.parquet
   2. The data for subsequent days would always be loaded as a new folder in that S3 bucket with that day's date.
      1. Example for Day 1, July 9th, 2023: s3://examplebucketname/foldername/20230709/anotherfolder/filename.parquet
      2. Example for Day 2, July 10th, 2023: s3://examplebucketname/foldername/20230710/anotherfolder/filename.parquet
3. Send a ticket to Platform Support requesting that we set up a new dataset using the S3 connection and .parquet files that you loaded.  Please include the S3 file path from step 1.