Skip to main content


Archives all the messages of a batch into a single message according to the selected archive format.

# Config fields, showing default values
label: ""
format: binary
path: ${!count("files")}-${!timestamp_unix_nano()}.txt

Some archive formats (such as tar, zip) treat each archive item (message part) as a file with a path. Since message parts only contain raw data a unique path must be generated for each part. This can be done by using function interpolations on the 'path' field as described here. For types that aren't file based (such as binary) the file field is ignored.

The resulting archived message adopts the metadata of the first message part of the batch.

The functionality of this processor depends on being applied across messages that are batched. You can find out more about batching in this doc.



The archiving format to apply.

Type: string
Default: "binary"
Options: tar, zip, binary, lines, json_array, concatenate.


The path to set for each message in the archive (when applicable). This field supports interpolation functions.

Type: string
Default: "${!count(\"files\")}-${!timestamp_unix_nano()}.txt"

# Examples
path: ${!count("files")}-${!timestamp_unix_nano()}.txt
path: ${!meta("kafka_key")}-${!json("id")}.json



Join the raw contents of each message into a single binary message.


Archive messages to a unix standard tape archive.


Archive messages to a zip file.


Archive messages to a binary blob format consisting of:

  • Four bytes containing number of messages in the batch (in big endian)
  • For each message part:
    • Four bytes containing the length of the message (in big endian)
    • The content of message


Join the raw contents of each message and insert a line break between each one.


Attempt to parse each message as a JSON document and append the result to an array, which becomes the contents of the resulting message.


If we had JSON messages in a batch each of the form:

{"doc":{"id":"foo","body":"hello world 1"}}

And we wished to tar archive them, setting their filenames to their respective unique IDs (with the extension .json), our config might look like this:

- archive:
format: tar
path: ${!json("")}.json