redis

Performs actions against Redis that aren't possible using a cache processor. Actions are performed for each message of a batch, where the contents are replaced with the result.

Common

# Common config fields, showing default values
label: ""
redis:
  url: tcp://localhost:6379
  operator: scard
  key: ""

Advanced

# All config fields, showing default values
label: ""
redis:
  url: tcp://localhost:6379
  kind: simple
  master: ""
  tls:
    enabled: false
    skip_cert_verify: false
    enable_renegotiation: false
    root_cas: ""
    root_cas_file: ""
    client_certs: []
  operator: scard
  key: ""
  retries: 3
  retry_period: 500ms
  parts: []

Operators

keys

Returns an array of strings containing all the keys that match the pattern specified by the key field.

scard

Returns the cardinality of a set, or 0 if the key does not exist.

sadd

Adds a new member to a set. Returns 1 if the member was added.

incrby

Increments the number stored at key by the message content. If the key does not exist, it is set to 0 before performing the operation. Returns the value of key after the increment.

Examples

Querying Cardinality

If given payloads containing a metadata field set_key it's possible to query and store the cardinality of the set for each message using a branch processor in order to augment rather than replace the message contents:

pipeline:
  processors:
    - branch:
        processors:
          - redis:
              url: TODO
              operator: scard
              key: ${! meta("set_key") }
        result_map: 'root.cardinality = this'

Running Total

If we have JSON data containing number of friends visited during covid 19:

{"name":"ash","month":"feb","year":2019,"friends_visited":10}
{"name":"ash","month":"apr","year":2019,"friends_visited":-2}
{"name":"bob","month":"feb","year":2019,"friends_visited":3}
{"name":"bob","month":"apr","year":2019,"friends_visited":1}

We can add a field that contains the running total number of friends visited:

{"name":"ash","month":"feb","year":2019,"friends_visited":10,"total":10}
{"name":"ash","month":"apr","year":2019,"friends_visited":-2,"total":8}
{"name":"bob","month":"feb","year":2019,"friends_visited":3,"total":3}
{"name":"bob","month":"apr","year":2019,"friends_visited":1,"total":4}

Using the incrby operator:

pipeline:
  processors:
    - branch:
        request_map: |
            root = this.friends_visited
            meta name = this.name
        processors:
          - redis:
              url: TODO
              operator: incrby
              key: ${! meta("name") }
        result_map: 'root.total = this'

Fields

url

The URL of the target Redis server. Database is optional and is supplied as the URL path. The scheme tcp is equivalent to redis.

Type: string
Default: "tcp://localhost:6379"

# Examples

url: :6397

url: localhost:6397

url: redis://localhost:6379

url: redis://:foopassword@redisplace:6379

url: redis://localhost:6379/1

url: redis://localhost:6379/1,redis://localhost:6380/1

kind

Specifies a simple, cluster-aware, or failover-aware redis client.

Type: string
Default: "simple"

# Examples

kind: simple

kind: cluster

kind: failover

master

Name of the redis master when kind is failover

Type: string
Default: ""

# Examples

master: mymaster

tls

Custom TLS settings can be used to override system defaults.

Troubleshooting

Some cloud hosted instances of Redis (such as Azure Cache) might need some hand holding in order to establish stable connections. Unfortunately, it is often the case that TLS issues will manifest as generic error messages such as "i/o timeout". If you're using TLS and are seeing connectivity problems consider setting enable_renegotiation to true, and ensuring that the server supports at least TLS version 1.2.

Type: object

tls.enabled

Whether custom TLS settings are enabled.

Type: bool
Default: false

tls.skip_cert_verify

Whether to skip server side certificate verification.

Type: bool
Default: false

tls.enable_renegotiation

Whether to allow the remote server to repeatedly request renegotiation. Enable this option if you're seeing the error message local error: tls: no renegotiation.

Type: bool
Default: false
Requires version 3.45.0 or newer

tls.root_cas

An optional root certificate authority to use. This is a string, representing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate.

Type: string
Default: ""

# Examples

root_cas: |-
  -----BEGIN CERTIFICATE-----
  ...
  -----END CERTIFICATE-----

tls.root_cas_file

An optional path of a root certificate authority file to use. This is a file, often with a .pem extension, containing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate.

Type: string
Default: ""

# Examples

root_cas_file: ./root_cas.pem

tls.client_certs

A list of client certificates to use. For each certificate either the fields cert and key, or cert_file and key_file should be specified, but not both.

Type: array
Default: []

# Examples

client_certs:
  - cert: foo
    key: bar

client_certs:
  - cert_file: ./example.pem
    key_file: ./example.key

tls.client_certs[].cert

A plain text certificate to use.

Type: string
Default: ""

tls.client_certs[].key

A plain text certificate key to use.

Type: string
Default: ""

tls.client_certs[].cert_file

The path to a certificate to use.

Type: string
Default: ""

tls.client_certs[].key_file

The path of a certificate key to use.

Type: string
Default: ""

operator

The operator to apply.

Type: string
Default: "scard"
Options: scard, sadd, incrby, keys.

key

A key to use for the target operator. This field supports interpolation functions.

Type: string
Default: ""

retries

The maximum number of retries before abandoning a request.

Type: int
Default: 3

retry_period

The time to wait before consecutive retry attempts.

Type: string
Default: "500ms"

parts

An optional array of message indexes of a batch that the processor should apply to. If left empty all messages are processed. This field is only applicable when batching messages at the input level.

Indexes can be negative, and if so the part will be selected from the end counting backwards starting from -1.

Type: array
Default: []