Skip to main content

aws_s3

Stores each item in an S3 bucket as a file, where an item ID is the path of the item within the bucket.

Introduced in version 3.36.0.

# Common config fields, showing default values
label: ""
aws_s3:
  bucket: ""
  content_type: application/octet-stream
  region: eu-west-1

It is not possible to atomically upload S3 objects exclusively when the target does not already exist, therefore this cache is not suitable for deduplication.

Credentials​

By default Benthos will use a shared credentials file when connecting to AWS services. It's also possible to set them explicitly at the component level, allowing you to transfer data across accounts. You can find out more in this document.

Fields​

bucket​

The S3 bucket to store items in.

Type: string
Default: ""

content_type​

The content type to set for each item.

Type: string
Default: "application/octet-stream"

force_path_style_urls​

Forces the client API to use path style URLs, which helps when connecting to custom endpoints.

Type: bool
Default: false

timeout​

The maximum period to wait on requests before abandoning it.

Type: string
Default: "5s"

retries​

The maximum number of retry attempts to make before abandoning a request.

Type: int
Default: 3

region​

The AWS region to target.

Type: string
Default: "eu-west-1"

endpoint​

Allows you to specify a custom endpoint for the AWS API.

Type: string
Default: ""

credentials​

Optional manual configuration of AWS credentials to use. More information can be found in this document.

Type: object

credentials.profile​

A profile from ~/.aws/credentials to use.

Type: string
Default: ""

credentials.id​

The ID of credentials to use.

Type: string
Default: ""

credentials.secret​

The secret for the credentials being used.

Type: string
Default: ""

credentials.token​

The token for the credentials being used, required when using short term credentials.

Type: string
Default: ""

credentials.role​

A role ARN to assume.

Type: string
Default: ""

credentials.role_external_id​

An external ID to provide when assuming a role.

Type: string
Default: ""