> For the complete documentation index, see [llms.txt](https://integrations.drivetrain.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://integrations.drivetrain.ai/integration-guide/readme/databases-and-warehouses/integrating-with-s3.md).
# Integrating with S3
Connect an S3 bucket to Drivetrain and ingest object storage data into financial models. Covers IAM user, access keys, and bucket policy.
[Prerequisites](#prerequisites)
[Amazon S3 Setup](#amazon-s3-setup)\
[Integration Methods](#integration-methods)\
[Method 1: Using Access Key and Secret](#method-1-using-access-key-and-secret)\
[Step 1: Create an IAM policy](#step-1-create-an-iam-policy)\
[Step 2: Create User](#step-2-create-user)\
[Step 3: Generate the Access key and secret](#step-3-generate-the-access-key-and-secret)\
[Step 4: Integration details](#step-4-integration-details)\
[Method 2: Using the IAM role](#method-2-using-iam-role)\
[Step 1: Create an IAM policy](#step-1-create-an-iam-policy-1)\
[Step 2: Create an IAM role](#step-2-create-an-iam-role)\
[Step 3: Adding permissions to the S3 bucket](#step-3-adding-permissions-to-the-s3-bucket)\
[Step 4: Integration details](#step-4-integration-details-1)
## Prerequisites
To connect your Amazon S3 bucket to Drivetrain, you need:
* An S3 bucket containing CSV, Excel (`.xlsx`, `.xls`), JSON or Parquet files
* Admin access in Drivetrain
* For private or encrypted buckets, an AWS account with the ability to grant Drivetrain permission and to read from the bucket
## Amazon S3 Setup
We recommend disabling Access Control Lists (ACLs) on each S3 bucket so that the bucket contents are controlled by the bucket's access control settings and not the original file owner's settings. For more information about disabling ACLs for your bucket, see [Amazon S3 documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html).
## Integration Methods
Method 1: Using the Access key and Secret
* Bucket name
* Folder path
* Access key
* Access secret
* AWS Region
Method 2: Using Roles (More secure)
* Bucket name
* Folder path
* AWS Region
* Role ARN
* External ID (if you set one)
## Method 1: Using Access Key and Secret
### Step 1: Create an IAM Policy
{% hint style="info" %}
You must create an IAM policy for both the IAM Role and Access Key and Secret approaches.
{% endhint %}
{% hint style="info" %}
For encrypted buckets, follow [Amazon S3 bucket instructions](https://aws.amazon.com/premiumsupport/knowledge-center/s3-bucket-access-default-encryption/) to modify the AWS KMS key's policy to grant Drivetrain permissions to download files from your encrypted bucket
{% endhint %}
1. Open your [Amazon IAM console](https://console.aws.amazon.com/iam/home#home).
2. Go to **Policies**, then click **Create Policy**
3. Go to the **JSON** tab
4. Copy the following policy and paste it into the visual editor. Replace `{your-bucket-name}` with the name of your S3 bucket. After that, click **Next: Tags**.
```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::{your-bucket-name}/*",
"arn:aws:s3:::{your-bucket-name}"
]
}
]
}
```
5\. ***(Optional)*** If you use a customer-managed KMS key, add the following policy to the Action section of the IAM policy to provide read access to the encrypted files.
```
Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
]
```
6. In the Add tags step, you can optionally add custom tags that will be associated with your bucket. Click **Next: Review**.
7. In the Review policy step, specify the name of your policy, for example "Drivetrain-S3-Access", then click **Create policy.**
### Step 2: Create User
1. Open your [Amazon IAM console](https://console.aws.amazon.com/iam/home#home).
2. Go to **Users**, then click **Create user**.
3. Enter the user name, then click **Next**.
4. Select the **Attach policies directly** option, then choose the "Drivetrain-S3-Access" policy you created in Step 1.
5. Click **Next**, then click **Create User.**
### Step 3: Generate the access key and secret
1. In the **Users** tab, open the User you created.
2. Go to the **Security credentials** tab and navigate to the **Access keys** section.
3. Click the **Create access key**.
4. From the **Use Case** options, select the **Third-party service** option and then click **Next**.
5. Enter a **Description tag** value and then click the **Create access key**.
6. Copy the **Access key** and **Secret access key** values. You will need them to configure the integration.
### Step 4: Integration details
You need these details to connect:
1. Bucket name
2. Folder path
3. Access key
4. Access secret
5. AWS Region
{% hint style="info" %}
To connect in Drivetrain, go to **Data** in the left navigation, click the **+** button and select **Amazon S3**. Click **Request access** and enter each detail above. If the form has no field for a detail, click **Add field** and enter a **Field name** and **Value**. Click **Submit credentials**. Drivetrain then completes the connection for you. If a connection form opens instead, enter the same details there.
{% endhint %}
## Method 2: Using IAM Role
### Step 1: Create an IAM Policy
Follow the same steps as in [Step 1](#step-1-create-an-iam-policy) of Method 1.
### Step 2: Create an IAM Role
1. Go to Roles, then click **Create role.**
2. Select **AWS account** > **Another AWS account**, then enter Drivetrain’s AWS Account ID, `865992467666`, in the **Account ID** field. Optionally select **Require external ID** and enter an external ID of your choice.
3. In the **Add permissions** step, select the "Drivetrain-S3-Access" policy you created, then click **Next.**
4. Enter the name Drivetrain-Integration check Trust policy settings and attached policy and click Create Role.
### Step 3: Adding permissions to the S3 bucket
This step is only needed if Drivetrain reads the bucket with its own role. If you share the Role ARN in Step 4, the policy from Step 1 on your role is enough. To assign permissions to your S3 bucket, follow the below steps:
1. Select the bucket to which you want to assign permissions.
2. Navigate to the **Permissions** tab.
3. Go to **Bucket Policy.**
4. Click **Edit**. Copy the below policy and paste it into the visual editor. Replace `{your-bucket-name}` with the name of your S3 bucket.
```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::865992467666:role/Drivetrain-Integration"
},
"Action": [
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::{your-bucket-name}/*",
"arn:aws:s3:::{your-bucket-name}"
]
}
]
}
```
5. Click 'Save' to apply the changes.
### Step 4: Integration details
You need these details to connect:
1. Bucket name.
2. Folder path.
3. AWS Region
4. Role ARN of the role you created in Step 2, for example `arn:aws:iam:::role/Drivetrain-Integration`
5. The external ID, if you set one
{% hint style="info" %}
To connect in Drivetrain, go to **Data** in the left navigation, click the **+** button and select **Amazon S3**. Click **Request access** and enter each detail above. If the form has no field for a detail, click **Add field** and enter a **Field name** and **Value**. Click **Submit credentials**. Drivetrain then completes the connection for you. If a connection form opens instead, enter the same details there.
{% endhint %}
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://integrations.drivetrain.ai/integration-guide/readme/databases-and-warehouses/integrating-with-s3.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
