Skip to main content

aws_s3

Stores each item in an S3 bucket as a file, where an item ID is the path of the item within the bucket.

Introduced in version 3.36.0.

# Common config fields, showing default values
label: ""
aws_s3:
bucket: ""
content_type: application/octet-stream
region: eu-west-1

It is not possible to atomically upload S3 objects exclusively when the target does not already exist, therefore this cache is not suitable for deduplication.

Credentials

By default Benthos will use a shared credentials file when connecting to AWS services. It's also possible to set them explicitly at the component level, allowing you to transfer data across accounts. You can find out more in this document.

Fields

bucket

The S3 bucket to store items in.

Type: string
Default: ""

content_type

The content type to set for each item.

Type: string
Default: "application/octet-stream"

force_path_style_urls

Forces the client API to use path style URLs, which helps when connecting to custom endpoints.

Type: bool
Default: false

timeout

The maximum period to wait on requests before abandoning it.

Type: string
Default: "5s"

retries

The maximum number of retry attempts to make before abandoning a request.

Type: int
Default: 3

region

The AWS region to target.

Type: string
Default: "eu-west-1"

endpoint

Allows you to specify a custom endpoint for the AWS API.

Type: string
Default: ""

credentials

Optional manual configuration of AWS credentials to use. More information can be found in this document.

Type: object

credentials.profile

A profile from ~/.aws/credentials to use.

Type: string
Default: ""

credentials.id

The ID of credentials to use.

Type: string
Default: ""

credentials.secret

The secret for the credentials being used.

Type: string
Default: ""

credentials.token

The token for the credentials being used, required when using short term credentials.

Type: string
Default: ""

credentials.role

A role ARN to assume.

Type: string
Default: ""

credentials.role_external_id

An external ID to provide when assuming a role.

Type: string
Default: ""