Bloblang Functions

Functions can be placed anywhere and allow you to extract information from your environment, generate values, or access data from the underlying message being mapped:

root.doc.id = uuid_v4()
root.doc.received_at = now()
root.doc.host = hostname()

Functions support both named and nameless style arguments:

root.values_one = range(start: 0, stop: this.max, step: 2)
root.values_two = range(0, this.max, 2)

General

count

The count function is a counter starting at 1 which increments after each time it is called. Count takes an argument which is an identifier for the counter, allowing you to specify multiple unique counters in your configuration.

Parameters

name <string> An identifier for the counter.

Examples

root = this
root.id = count("bloblang_function_example")

# In:  {"message":"foo"}
# Out: {"id":1,"message":"foo"}

# In:  {"message":"bar"}
# Out: {"id":2,"message":"bar"}

deleted

A function that returns a result indicating that the mapping target should be deleted. Deleting, also known as dropping, messages will result in them being acknowledged as successfully processed to inputs in a Benthos pipeline. For more information about error handling patterns read here.

Examples

root = this
root.bar = deleted()

# In:  {"bar":"bar_value","baz":"baz_value","foo":"foo value"}
# Out: {"baz":"baz_value","foo":"foo value"}

Since the result is a value it can be used to do things like remove elements of an array within map_each.

root.new_nums = this.nums.map_each(num -> if num < 10 { deleted() } else { num - 10 })

# In:  {"nums":[3,11,4,17]}
# Out: {"new_nums":[1,7]}

ksuid

Generates a new ksuid each time it is invoked and prints a string representation.

Examples

root.id = ksuid()

nanoid

Generates a new nanoid each time it is invoked and prints a string representation.

Parameters

length <(optional) integer> An optional length.
alphabet <(optional) string> An optional custom alphabet to use for generating IDs. When specified the field length must also be present.

Examples

root.id = nanoid()

It is possible to specify an optional length parameter.

root.id = nanoid(54)

It is also possible to specify an optional custom alphabet after the length parameter.

root.id = nanoid(54, "abcde")

random_int

Generates a non-negative pseudo-random 64-bit integer. An optional integer argument can be provided in order to seed the random number generator.

Parameters

seed <query expression, default {"Value":0}> A seed to use, if a query is provided it will only be resolved once during the lifetime of the mapping.

Examples

root.first = random_int()
root.second = random_int(1)

It is possible to specify a dynamic seed argument, in which case the argument will only be resolved once during the lifetime of the mapping.

root.first = random_int(timestamp_unix_nano())

range

The range function creates an array of integers following a range between a start, stop and optional step integer argument. If the step argument is omitted then it defaults to 1. A negative step can be provided as long as stop < start.

Parameters

start <integer> The start value.
stop <integer> The stop value.
step <integer, default 1> The step value.

Examples

root.a = range(0, 10)
root.b = range(start: 0, stop: this.max, step: 2) # Using named params
root.c = range(0, -this.max, -2)

# In:  {"max":10}
# Out: {"a":[0,1,2,3,4,5,6,7,8,9],"b":[0,2,4,6,8],"c":[0,-2,-4,-6,-8]}

throw

Throws an error similar to a regular mapping error. This is useful for abandoning a mapping entirely given certain conditions.

Parameters

why <string> A string explanation for why an error was thrown, this will be added to the resulting error message.

Examples

root.doc.type = match {
  this.exists("header.id") => "foo"
  this.exists("body.data") => "bar"
  _ => throw("unknown type")
}
root.doc.contents = (this.body.content | this.thing.body)

# In:  {"header":{"id":"first"},"thing":{"body":"hello world"}}
# Out: {"doc":{"contents":"hello world","type":"foo"}}

# In:  {"nothing":"matches"}
# Out: Error("failed assignment (line 1): unknown type")

uuid_v4

Generates a new RFC-4122 UUID each time it is invoked and prints a string representation.

Examples

root.id = uuid_v4()

Message Info

batch_index

Returns the index of the mapped message within a batch. This is useful for applying maps only on certain messages of a batch.

Examples

root = if batch_index() > 0 { deleted() }

batch_size

Returns the size of the message batch.

Examples

root.foo = batch_size()

content

Returns the full raw contents of the mapping target message as a byte array. When mapping to a JSON field the value should be encoded using the method encode, or cast to a string directly using the method string, otherwise it will be base64 encoded by default.

Examples

root.doc = content().string()

# In:  {"foo":"bar"}
# Out: {"doc":"{\"foo\":\"bar\"}"}

error

If an error has occurred during the processing of a message this function returns the reported cause of the error. For more information about error handling patterns read here.

Examples

root.doc.error = error()

errored

Returns a boolean value indicating whether an error has occurred during the processing of a message. For more information about error handling patterns read here.

Examples

root.doc.status = if errored() { 400 } else { 200 }

json

Returns the value of a field within a JSON message located by a dot path argument. This function always targets the entire source JSON document regardless of the mapping context.

Parameters

path <string, default ""> An optional dot path identifying a field to obtain.

Examples

root.mapped = json("foo.bar")

# In:  {"foo":{"bar":"hello world"}}
# Out: {"mapped":"hello world"}

The path argument is optional and if omitted the entire JSON payload is returned.

root.doc = json()

# In:  {"foo":{"bar":"hello world"}}
# Out: {"doc":{"foo":{"bar":"hello world"}}}

meta

Returns the value of a metadata key from the input message. Since values are extracted from the read-only input message they do NOT reflect changes made from within the map. In order to query metadata mutations made within a mapping use the root_meta function. This function supports extracting metadata from other messages of a batch with the from method.

Parameters

key <string, default ""> An optional key of a metadata value to obtain.

Examples

root.topic = meta("kafka_topic")

If the target key does not exist an error is thrown, allowing you to use coalesce or catch methods to fallback to other queries.

root.topic = meta("nope") | meta("also nope") | "default"

The parameter is optional and if omitted the entire metadata contents are returned as an object.

root.all_metadata = meta()

root_meta

BETA: This function is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Returns the value of a metadata key from the new message being created. Changes made to metadata during a mapping will be reflected by this function.

Parameters

key <string, default ""> An optional key of a metadata value to obtain.

Examples

root.topic = root_meta("kafka_topic")

If the target key does not exist an error is thrown, allowing you to use coalesce or catch methods to fallback to other queries.

root.topic = root_meta("nope") | root_meta("also nope") | "default"

The parameter is optional and if omitted the entire metadata contents are returned as an object.

root.all_metadata = root_meta()

Environment

env

Returns the value of an environment variable, or an empty string if the environment variable does not exist.

Parameters

name <string> The name of an environment variable.

Examples

root.thing.key = env("key")

file

BETA: This function is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Reads a file and returns its contents. Relative paths are resolved from the directory of the process executing the mapping.

Parameters

path <string> The path of the target file.

Examples

root.doc = file(env("BENTHOS_TEST_BLOBLANG_FILE")).parse_json()

# In:  {}
# Out: {"doc":{"foo":"bar"}}

hostname

Returns a string matching the hostname of the machine running Benthos.

Examples

root.thing.host = hostname()

now

Returns the current timestamp as a string in ISO 8601 format with the local timezone. Use the method format_timestamp in order to change the format and timezone.

Examples

root.received_at = now()

root.received_at = now().format_timestamp("Mon Jan 2 15:04:05 -0700 MST 2006", "UTC")

timestamp_unix

Returns the current unix timestamp in seconds.

Examples

root.received_at = timestamp_unix()

timestamp_unix_nano

Returns the current unix timestamp in nanoseconds.

Examples

root.received_at = timestamp_unix_nano()

Deprecated

timestamp

Returns the current time in a custom format specified by the argument. The format is defined by showing how the reference time, defined to be Mon Jan 2 15:04:05 -0700 MST 2006 would be displayed if it were the value.

A fractional second is represented by adding a period and zeros to the end of the seconds section of layout string, as in 15:04:05.000 to format a time stamp with millisecond precision. This has been deprecated in favour of the new now function.

Parameters

format <string, default "Mon Jan 2 15:04:05 -0700 MST 2006"> The format to print as.

Examples

root.received_at = timestamp("15:04:05")

timestamp_utc

The equivalent of timestamp except the time is printed as UTC instead of the local timezone. This has been deprecated in favour of the new now function.

Parameters

format <string, default "Mon Jan 2 15:04:05 -0700 MST 2006"> The format to print as.

Examples

root.received_at = timestamp_utc("15:04:05")