Skip to main content

hdfs

Reads files from a HDFS directory, where each discrete file will be consumed as a single message payload.

# Config fields, showing default values
input:
  label: ""
  hdfs:
    hosts:
      - localhost:9000
    user: benthos_hdfs
    directory: ""

Metadata​

This input adds the following metadata fields to each message:

- hdfs_name
- hdfs_path

You can access these metadata fields using function interpolation.

Fields​

hosts​

A list of target host addresses to connect to.

Type: array
Default: ["localhost:9000"]

user​

A user ID to connect as.

Type: string
Default: "benthos_hdfs"

directory​

The directory to consume from.

Type: string
Default: ""