Configuration Framework
Zeek includes a configuration framework that allows updating script options at
runtime. This functionality consists of an option
declaration in
the Zeek language, configuration files that enable changing the value of
options at runtime, option-change callbacks to process updates in your Zeek
scripts, a couple of script-level functions to manage config settings directly,
and a log file (config.log
) that contains information about every
option value change according to Config::Info
.
Introduction
The configuration framework provides an alternative to using Zeek script constants to store various Zeek settings.
While traditional constants work well when a value is not expected to change at
runtime, they cannot be used for values that need to be modified occasionally.
While a redef
allows a re-definition of an already defined constant
in Zeek, these redefinitions can only be performed when Zeek first starts.
Afterwards, constants can no longer be modified.
However, it is clearly desirable to be able to change at runtime many of the configuration options that Zeek offers. Restarting Zeek can be time-consuming and causes it to lose all connection state and knowledge that it accumulated. Zeek’s configuration framework solves this problem.
Declaring Options
The option
keyword allows variables to be declared as configuration
options:
module Test;
export {
option my_networks: set[subnet] = {};
option enable_feature = F;
option hostname = "testsystem";
option timeout_after = 1min;
option my_ports: vector of port = {};
}
Options combine aspects of global variables and constants. Like global variables, options cannot be declared inside a function, hook, or event handler. Like constants, options must be initialized when declared (the type can often be inferred from the initializer but may need to be specified when ambiguous). The value of an option can change at runtime, but options cannot be assigned a new value using normal assignments.
The initial value of an option can be redefined with a redef
declaration just like for global variables and constants. However, there is no
need to specify the &redef
attribute in the declaration of an
option. For example, given the above option declarations, here are possible
redefs that work anyway:
redef Test::enable_feature = T;
redef Test::my_networks += { 10.1.0.0/16, 10.2.0.0/16 };
Changing Options
The configuration framework facilitates reading in new option values from external files at runtime. Configuration files contain a mapping between option names and their values. Each line contains one option assignment, formatted as follows:
[option name][tab/spaces][new value]
Lines starting with #
are comments and ignored.
You register configuration files by adding them to
Config::config_files
, a set of filenames. Simply say something like
the following in local.zeek
:
redef Config::config_files += { "/path/to/config.dat" };
Zeek will then monitor the specified file continuously for changes. For example, editing a line containing:
Test::enable_feature T
to the config file while Zeek is running will cause it to automatically update the option’s value in the scripting layer. The next time your code accesses the option, it will see the new value.
Note
The config framework is clusterized. In a cluster configuration, only the manager node watches the specified configuration files, and relays option updates across the cluster.
Config File Formatting
The formatting of config option values in the config file is not the same as in
Zeek’s scripting language. Keep an eye on the reporter.log
for warnings
from the config reader in case of incorrectly formatted values, which it’ll
generally ignore when encountered. The following table summarizes supported
types and their value representations:
Data Type |
Sample Config File Entry |
Comments |
---|---|---|
|
Plain IPv4 or IPv6 address, as in Zeek. No |
|
|
|
|
|
Plain, nonnegative integer. |
|
|
Plain double number. |
|
|
Plain enum string. |
|
|
Plain integer. |
|
|
Always in epoch seconds, with optional fraction of seconds. Never includes a time unit. |
|
|
The regex pattern, within forward-slash characters. |
|
|
Port number with protocol, as in Zeek. When the protocol part is missing,
Zeek interprets it as |
|
|
The set members, formatted as per their own type, separated by commas. For an empty set, use an empty string: just follow the option name with whitespace. Sets with multiple index types (e.g. |
|
|
Plain string, no quotation marks. Given quotation marks become part of
the string. Everything after the whitespace separator delineating the
option name becomes the string. Saces and special characters are fine.
Backslash characters (e.g. |
|
|
Plain subnet, as in Zeek. |
|
|
Always in epoch seconds, with optional fraction of seconds. Never includes a time unit. |
|
|
The set members, formatted as per their own type, separated by commas. For an empty vector, use an empty string: just follow the option name with whitespace. |
This leaves a few data types unsupported, notably tables and records. If you
require these, build up an instance of the corresponding type manually (perhaps
from a separate input framework file) and then call
Config::set_value
to update the option:
module Test;
export {
option host_port: table[addr] of port = {};
}
event zeek_init() {
local t: table[addr] of port = { [10.0.0.2] = 123/tcp };
Config::set_value("Test::host_port", t);
}
Regardless of whether an option change is triggered by a config file or via
explicit Config::set_value
calls, Zeek always logs the change to
config.log
. A sample entry:
#fields ts id old_value new_value location
#types time string string string string
1608167352.498872 Test::a_count 42 3 config.txt
Mentioning options repeatedly in the config files leads to multiple update
events; the last entry “wins”. Mentioning options that do not correspond to
existing options in the script layer is safe, but triggers warnings in
reporter.log
:
warning: config.txt/Input::READER_CONFIG: Option 'an_unknown' does not exist. Ignoring line.
Internally, the framework uses the Zeek input framework to learn about config
changes. If you inspect the configuration framework scripts, you will notice
that the scripts simply catch input framework events and call
Config::set_value
to set the relevant option to the new value. If
you want to change an option in your scripts at runtime, you can likewise call
Config::set_value
directly from a script (in a cluster
configuration, this only needs to happen on the manager, as the change will be
automatically sent to all other nodes in the cluster).
Note
The input framework is usually very strict about the syntax of input files, but that is not the case for configuration files. These require no header lines, and both tabs and spaces are accepted as separators. A custom input reader, specifically for reading config files, facilitates this.
Tip
The gory details of option-parsing reside in Ascii::ParseValue()
in
src/threading/formatters/Ascii.cc
and Value::ValueToVal
in
src/threading/SerialTypes.cc
in the Zeek core.
Change Handlers
A change handler is a user-defined function that Zeek calls each time an option
value changes. This allows you to react programmatically to option changes. The
following example shows how to register a change handler for an option that has
a data type of addr
(for other data types, the return type and
second parameter data type must be adjusted accordingly):
module Test;
export {
option testaddr = 127.0.0.1;
}
# Note: the data type of 2nd parameter and return type must match
function change_addr(id: string, new_value: addr): addr
{
print fmt("Value of %s changed from %s to %s", id, testaddr, new_value);
return new_value;
}
event zeek_init()
{
Option::set_change_handler("Test::testaddr", change_addr);
}
Immediately before Zeek changes the specified option value, it invokes any registered change handlers. The value returned by the change handler is the value Zeek assigns to the option. This allows, for example, checking of values to reject invalid input (the original value can be returned to override the change).
Note
Option::set_change_handler
expects the name of the option to
invoke the change handler for, not the option itself. Also, that name
includes the module name, even when registering from within the module.
It is possible to define multiple change handlers for a single option. In this
case, the change handlers are chained together: the value returned by the first
change handler is the “new value” seen by the next change handler, and so on.
The built-in function Option::set_change_handler
takes an optional
third argument that can specify a priority for the handlers.
A change handler function can optionally have a third argument of type string.
When a config file triggers a change, then the third argument is the pathname
of the config file. When the Config::set_value
function triggers a
change, then the third argument of the change handler is the value passed to
the optional third argument of the Config::set_value
function.
Tip
Change handlers are also used internally by the configuration framework. If
you look at the script-level source code of the config framework, you can see
that change handlers log the option changes to config.log
.
When Change Handlers Trigger
Change handlers often implement logic that manages additional internal state. For example, depending on a performance toggle option, you might initialize or clean up a caching structure. In such scenarios you need to know exactly when and whether a handler gets invoked. The following hold:
When no config files get registered in
Config::config_files
, change handlers do not run.When none of any registered config files exist on disk, change handlers do not run.
That is, change handlers are tied to config files, and don’t automatically run with the option’s default values.
When a config file exists on disk at Zeek startup, change handlers run with the file’s config values.
When the config file contains the same value the option already defaults to, its change handlers are invoked anyway.
zeek_init
handlers run before any change handlers — i.e., they run with the options’ default values.Since the config framework relies on the input framework, the input framework’s inherent asynchrony applies: you can’t assume when exactly an option change manifests in the code.
If your change handler needs to run consistently at startup and when options
change, you can call the handler manually from zeek_init
when you
register it. That way, initialization code always runs for the option’s default
value, and also for any new values.
module Test;
export {
option use_cache = T;
}
function use_cache_hdlr(id: string, new_value: bool): bool
{
if ( new_value ) {
# Ensure caching structures are set up properly
}
return new_value;
}
event zeek_init()
{
use_cache_hdlr("Test::use_cache", use_cache);
Option::set_change_handler("Test::use_cache", use_cache_hdlr);
}