Skip to main content

cache

Stores each message in a cache.

# Common config fields, showing default values
output:
label: ""
cache:
target: ""
key: ${!count("items")}-${!timestamp_unix_nano()}
max_in_flight: 1

Caches are configured as resources and can target any of the following types:

The target field must point to a configured cache like follows:

output:
cache:
target: foo
key: ${!json("document.id")}
cache_resources:
- label: foo
memcached:
addresses:
- localhost:11211
ttl: 60

In order to create a unique key value per item you should use function interpolations described here. When sending batched messages the interpolations are performed per message part.

Performance

This output benefits from sending multiple messages in flight in parallel for improved performance. You can tune the max number of in flight messages with the field max_in_flight.

Fields

target

The target cache to store messages in.

Type: string
Default: ""

key

The key to store messages by, function interpolation should be used in order to derive a unique key for each message. This field supports interpolation functions.

Type: string
Default: "${!count(\"items\")}-${!timestamp_unix_nano()}"

# Examples
key: ${!count("items")}-${!timestamp_unix_nano()}
key: ${!json("doc.id")}
key: ${!meta("kafka_key")}

ttl

The TTL of each individual item as a duration string. After this period an item will be eligible for removal during the next compaction. Not all caches support per-key TTLs, and those that do not will fall back to their generally configured TTL setting. This field supports interpolation functions.

Type: string
Default: ""
Requires version 3.33.0 or newer

# Examples
ttl: 60s
ttl: 5m
ttl: 36h

max_in_flight

The maximum number of messages to have in flight at a given time. Increase this to improve throughput.

Type: int
Default: 1