Skip to main content

s3

DEPRECATED

This component is deprecated and will be removed in the next major version release. Please consider moving onto alternative components.

Stores each item in an S3 bucket as a file, where an item ID is the path of the item within the bucket.

# Common config fields, showing default values
label: ""
s3:
  bucket: ""
  content_type: application/octet-stream
  region: eu-west-1

Alternatives​

This cache has been renamed to aws_s3.

It is not possible to atomically upload S3 objects exclusively when the target does not already exist, therefore this cache is not suitable for deduplication.

Credentials​

By default Benthos will use a shared credentials file when connecting to AWS services. It's also possible to set them explicitly at the component level, allowing you to transfer data across accounts. You can find out more in this document.

Fields​

bucket​

The S3 bucket to store items in.

Type: string
Default: ""

content_type​

The content type to set for each item.

Type: string
Default: "application/octet-stream"

force_path_style_urls​

Forces the client API to use path style URLs, which helps when connecting to custom endpoints.

Type: bool
Default: false

timeout​

The maximum period to wait on requests before abandoning it.

Type: string
Default: "5s"

retries​

The maximum number of retry attempts to make before abandoning a request.

Type: int
Default: 3

region​

The AWS region to target.

Type: string
Default: "eu-west-1"

endpoint​

Allows you to specify a custom endpoint for the AWS API.

Type: string
Default: ""

credentials​

Optional manual configuration of AWS credentials to use. More information can be found in this document.

Type: object

credentials.profile​

A profile from ~/.aws/credentials to use.

Type: string
Default: ""

credentials.id​

The ID of credentials to use.

Type: string
Default: ""

credentials.secret​

The secret for the credentials being used.

Type: string
Default: ""

credentials.token​

The token for the credentials being used, required when using short term credentials.

Type: string
Default: ""

credentials.role​

A role ARN to assume.

Type: string
Default: ""

credentials.role_external_id​

An external ID to provide when assuming a role.

Type: string
Default: ""