We’re excited to announce that Amazon S3 is now available as a destination in Weld! This new feature enables you to sync data into your S3 buckets, giving you complete control over your data infrastructure.
Why use Amazon S3?
Amazon S3 is one of the most widely used cloud storage solutions, offering scalability, security, and seamless integration with analytics and AI platforms. With S3, you can:
- Centralize your data workflows by storing structured and unstructured data in one place.
- Use it as part of a data lake strategy, integrating with tools like Databricks for advanced analytics.
- Reduce costs with flexible storage options tailored to your needs.
- Improve security and compliance with enterprise-grade encryption and access controls.
How to set up Amazon S3 as a destination in Weld
Getting started is simple. Follow these steps to connect your S3 bucket to Weld:
Step 1: Create an S3 bucket
Set up an empty Amazon S3 bucket where your data will be stored.
Step 2: Create an IAM policy for Weld
- Go to the Amazon IAM Console and create a new policy.
- Define permissions for Weld to write data to your bucket.
Step 3: Create a user for Weld
- Add a new IAM user with programmatic access.
- Attach the Weld-S3-Access policy.
- Copy the Access Key ID and Secret Access Key for the next step.
Step 4: Authenticate Weld
- Enter your Access Key ID, Secret Key, and S3 bucket name in Weld.
- Choose your AWS region.
- Click Connect, and you’re ready to go!
For detailed setup instructions, check out our step-by-step guide.
