Skip to main content

cache

Stores each message in a cache.

# Common config fields, showing default values
output:
  label: ""
  cache:
    target: ""
    key: ${!count("items")}-${!timestamp_unix_nano()}
    max_in_flight: 1

Caches are configured as resources and can target any of the following types:

The target field must point to a configured cache like follows:

output:
  cache:
    target: foo
    key: ${!json("document.id")}


cache_resources:
  - label: foo
    memcached:
      addresses:
        - localhost:11211
      ttl: 60

In order to create a unique key value per item you should use function interpolations described here. When sending batched messages the interpolations are performed per message part.

Performance​

This output benefits from sending multiple messages in flight in parallel for improved performance. You can tune the max number of in flight messages with the field max_in_flight.

Fields​

target​

The target cache to store messages in.

Type: string
Default: ""

key​

The key to store messages by, function interpolation should be used in order to derive a unique key for each message. This field supports interpolation functions.

Type: string
Default: "${!count(\"items\")}-${!timestamp_unix_nano()}" 

# Examples


key: ${!count("items")}-${!timestamp_unix_nano()}


key: ${!json("doc.id")}


key: ${!meta("kafka_key")}

ttl​

The TTL of each individual item as a duration string. After this period an item will be eligible for removal during the next compaction. Not all caches support per-key TTLs, and those that do not will fall back to their generally configured TTL setting. This field supports interpolation functions.

Type: string
Default: ""
Requires version 3.33.0 or newer 

# Examples


ttl: 60s


ttl: 5m


ttl: 36h

max_in_flight​

The maximum number of messages to have in flight at a given time. Increase this to improve throughput.

Type: int
Default: 1