Attributes
The Zeek scripting language supports customization of many language elements via attributes. For example, attributes can ensure that a function gets invoked whenever you modify a table, automatically expire elements from a set, or tell the logging framework which record fields you’d like it to write. Zeek features the following attributes:
Name |
Description |
---|---|
Redefine a global constant or extend a type. |
|
Specify priority for event handler or hook. |
|
Mark a record field as to be written to a log. |
|
Allow a record field value to be missing. |
|
Specify a default value. |
|
Specify a default value for tables with insert behavior. |
|
Specify a function to call for each |
|
Same as |
|
Specify a function to call when container element expires. |
|
Specify a read timeout interval. |
|
Specify a write timeout interval. |
|
Specify a creation timeout interval. |
|
Specify a function to call on set/table changes |
|
Open file in raw mode (chars. are not escaped). |
|
Used internally for reporter framework events. |
|
Used by input framework for |
|
Used for table persistence/synchronization. |
|
Used for table persistence/synchronization. |
|
Used for table persistence/synchronization. |
|
Used for predictable member iteration of tables and sets. |
|
Marks an identifier as deprecated. |
|
Suppress “used before defined” warnings from |
|
Suppress lack-of-use warnings from |
|
Annotates event handlers and hooks with event groups. |
Warning
A confusing pitfall can be mistaking that attributes bind to a variable or a type, where in reality they bind to a value. Example:
global my_table: table[count] of string &create_expire=1sec;
event zeek_init()
{
my_table = table();
my_table[1] = "foo";
}
In the above, the re-assignment of my_table
will also drop the original
value’s &create_expire
and no entries will ever be expired
from my_table
. The alternate way of re-assignment that creates a new
table value with the expected attribute would be:
my_table = table() &create_expire=1sec;
Here is a more detailed explanation of each attribute:
&redef
Allows use of a redef
to redefine initial values of
global variables (i.e., variables declared either global
or const
). Example:
const clever = T &redef;
global cache_size = 256 &redef;
Note that a variable declared global
can also have its value changed
with assignment statements (doesn’t matter if it has the &redef
attribute or not).
&priority
Specifies the execution priority (as a signed integer) of a hook or
event handler. Higher values are executed before lower ones. The
default value is 0
. Example:
event zeek_init() &priority=10
{
print "high priority";
}
&log
When a record
field has the &log
attribute, this field is
included as a column in the log stream associated with the record type. This
association happens with Log::create_stream
and commonly looks as
follows:
redef enum Log::ID += { LOG };
type Info: record {
ts: time &log &default=network_time();
id: conn_id &log;
msg: string &log;
hidden: count &default=0; # This is not logged.
};
event zeek_init() {
Log::create_stream(LOG, [$columns=Info, $path="example"]);
}
The log stream above will have the columns ts
, id
and msg
.
When &log
is placed at the end of a record type declaration, all fields
listed in the declaration will have the &log
attribute implicitly.
type conn_id: record {
orig_h: addr;
orig_p: port;
resp_h: addr;
resp_p: port;
} &log;
Fields added to such a record types later on using redef
need to
explicitly specify &log
again, however.
&optional
Allows a record field value to be missing. Zeek allows such fields to remain
uninitialized and unassigned, and to have assigned values removed via
delete
.
In this example, the record could be instantiated with either
myrec($a=127.0.0.1)
or myrec($a=127.0.0.1, $b=80/tcp)
:
type myrec: record { a: addr; b: port &optional; };
The ?$
operator can be used to check if a record field has a value or
not (it returns a bool
value of T
if the field has a value,
and F
if not).
&default
Specifies a default value for a record field, container element, or a function/hook/event parameter.
In this example, the record could be instantiated with either
myrec($a=5, $c=3.14)
or myrec($a=5, $b=53/udp, $c=3.14)
:
type myrec: record { a: count; b: port &default=80/tcp; c: double; };
In this example, the table will return the string "foo"
for any
attempted access to a non-existing index:
global mytable: table[count] of string &default="foo";
In addition to constant values as shown above, the &default
attribute
also accepts arbitrary Zeek expressions. For example, arithmetic expressions and
function calls are possible:
type Info: record {
ts: time &log &default=network_time();
ts_ms: double &log &default=time_to_double(network_time()) * 1000;
};
The expressions are evaluated whenever a new record is instantiated.
On tables, the &default
attribute can further be set to a function
(including an anonymous lambda function), which will be invoked for any read access
to a non-existing index to generate a substitute result. The signature of such a default function
has to match with the index and value types of the given table. Below, a default
function for a table with a composite index and value type of string
is shown.
The arguments for the function call, c
and s
below, are populated with
the values used for the index:
function table_default(c: count, s: string): string {
return fmt("unknown-%s-%s", c, s);
}
global mytable: table[count, string] of string &default=table_default;
print mytable[0, "a"];
Using an anonymous function instead looks as follows:
global mytable: table[count, string] of string &default=function(c: count, s: string): string {
return fmt("unknown-%s-%s", c, s);
};
print mytable[0, "a"];
The output of both these examples is unknown-0-a
.
A common usage pattern of the &default
attribute in Zeek’s base
scripts is to format a default textual representation for unknown protocol
values that are otherwise mapped to textual descriptions.
The following excerpt is from base/protocols/dns/consts.zeek
mapping numeric DNS query types to their textual representation. A default
function is used to produce a string containing the numeric value of query types:
## Mapping of DNS query type codes to human readable string
## representation.
const query_types = {
[1] = "A",
[2] = "NS",
[3] = "MD",
[4] = "MF",
[5] = "CNAME",
# many many more ...
[65422] = "XPF",
[65521] = "INTEGRITY",
} &default = function(n: count): string { return fmt("query-%d", n); };
Note that when accessing a non-existing index, the created default value will
not be inserted into the table. The following script will output foo
,
but the table remains empty. The second print statement outputs 0
:
global mytable: table[count] of string &default="foo";
print mytable[0];
print |mytable|;
For inserting the created default value into a table, the &default_insert
attribute can be used instead.
When used with function/hook/event parameters, all of the parameters
with the &default
attribute must come after all other parameters.
For example, the following function could be called either as myfunc(5)
or as myfunc(5, 53/udp)
:
function myfunc(a: count, b: port &default=80/tcp)
{
print a, b;
}
&default_insert
New in version 6.1.
This attribute is only applicable to tables. &default_insert
provides the same functionality as table’s &default
but with the addition
that upon access to a non-existing index, the created value will be inserted
into the table. For complex value types like tables or record types used for
tracking further state, &default_insert
is often more useful and
efficient than &default
.
&add_func
Can be applied to an identifier with &redef to specify a function to
be called any time a redef <id> += ...
declaration is parsed. The
function takes two arguments of the same type as the identifier, the first
being the old value of the variable and the second being the new
value given after the +=
operator in the redef
declaration. The
return value of the function will be the actual new value of the
variable after the “redef” declaration is parsed.
&delete_func
Same as &add_func
, except for redef
declarations
that use the -=
operator.
&expire_func
Called right before a container element expires. The function’s first
argument is of the same type as the container it is associated with.
The function then takes a variable number of arguments equal to the
number of indexes in the container. For example, for a
table[string,string] of count
the expire function signature is:
function(t: table[string, string] of count, s: string, s2: string): interval
The return value is an interval
indicating the amount of
additional time to wait before expiring the container element at the
given index (which will trigger another execution of this function).
&read_expire
Specifies a read expiration timeout for container elements. That is, the element expires after the given amount of time since the last time it has been read. Note that a write also counts as a read.
&write_expire
Specifies a write expiration timeout for container elements. That is, the element expires after the given amount of time since the last time it has been written.
&create_expire
Specifies a creation expiration timeout for container elements. That is, the element expires after the given amount of time since it has been inserted into the container, regardless of any reads or writes.
Note
In order to support expiration timeouts, Zeek associates a timer with each container that weeds out stale entries. For containers with many members, Zeek needs to keep an eye on the amount of effort spent expiring elements. It does this via three configurable properties:
table_expire_interval
specifies how frequently Zeek checks a container’s members. The interval establishes an upper bound on how long it may take Zeek to react to an element’s expiration.table_incremental_step
specifies how many members Zeek checks in one batch.table_expire_delay
interval specifies how long Zeek waits until it processes the next batch of members.
&on_change
Called right after a change has been applied to a container. The function’s
first argument is of the same type as the container it is associated with,
followed by a TableChange
record which specifies the type of change
that happened. The function then takes a variable number of arguments equal to
the number of indexes in the container, followed by an argument for the value
of the container (if the container has a value) For example, for a
table[string,string] of count
the &on_change
function signature is:
function(t: table[string, string] of count, tpe: TableChange,
s: string, s2: string, val: count)
For a set[count]
the function signature is:
function(s: set[count], tpe: TableChange, c: count)
The passed value specifies the state of a value before the change, where this makes sense. In case a element is changed, removed, or expired, the passed value will be the value before the change, removal, or expiration. When an element is added, the passed value will be the value of the added element (since no old element existed).
Note that the &on_change
function is only called when the container itself
is modified (due to an assignment, delete operation, or expiry). When a
container contains a complex element (like a record, set, or vector), changes
to these complex elements are not propagated back to the parent. For example,
in this example the change_function
for the table will only be called once,
when s
is inserted, but it will not be called when s
is changed:
local t: table[string] of set[string] &on_change=change_function;
local s: set[string] = set();
t["s"] = s; # change_function of t is called
add s["a"]; # change_function of t is _not_ called.
Also note that the &on_change
function of a container will not be called
when the container is already executing its &on_change
function. Thus,
writing an &on_change
function like this is supported and will not lead to
a infinite loop:
local t: table[string] of set[string] &on_change=change_function;
function change_function(t: table[string, int] of count, tpe: TableChange,
idxa: string, idxb: int, val: count)
{
t[idxa, idxb] = val+1;
}
&raw_output
Opens a file in raw mode, i.e., non-ASCII characters are not escaped.
&error_handler
Internally set on the events that are associated with the reporter
framework: reporter_info
, reporter_warning
, and
reporter_error
. It prevents any handlers of those events
from being able to generate reporter messages that go through any of
those events (i.e., it prevents an infinite event recursion). Instead,
such nested reporter messages are output to stderr.
&type_column
Used by the input framework. It can be used on columns of type
port
(such a column only contains the port number) and
specifies the name of an additional column in
the input file which specifies the protocol of the port (tcp/udp/icmp).
In the following example, the input file would contain four columns
named ip
, srcp
, proto
, and msg
:
type Idx: record {
ip: addr;
};
type Val: record {
srcp: port &type_column = "proto";
msg: string;
};
&backend
Used for persisting tables/sets and/or synchronizing them over a cluster.
This attribute binds a table to a Broker store. Changes to the table are sent to the Broker store, and changes to the Broker store are applied back to the table.
Since Broker stores are synchronized over a cluster, this sends table changes to all other nodes in the cluster. When using a persistent Broker store backend, the content of the tables/sets will be restored on startup.
This attribute expects the type of backend you want to use for the table. For example, to bind a table to a memory-backed Broker store, use:
global t: table[string] of count &backend=Broker::MEMORY;
&broker_store
This attribute is similar to &backend
in allowing a Zeek table to
bind to a Broker store. It differs from &backend
as this attribute
allows you to specify the Broker store you want to bind, without creating it.
Use this if you want to bind a table to a Broker store with special options.
Example:
global teststore: opaque of Broker::Store;
global t: table[string] of count &broker_store="teststore";
event zeek_init()
{
teststore = Broker::create_master("teststore");
}
&broker_allow_complex_type
By default only tables containing atomic types can be bound to Broker stores.
Specifying this attribute before &backend
or &broker_store
disables this safety feature and allows complex types to be stored in a Broker backed
table.
Warning
Storing complex types in Broker backed store comes with severe restrictions. When you modify a stored complex type after inserting it into a table, that change in a stored complex type will not propagate to Broker. Hence to send out the new value, so that it will be persisted/synchronized over the cluster, you will have to re-insert the complex type into the local zeek table.
For example:
type testrec: record {
a: count;
};
global t: table[string] of testrec &broker_allow_complex_type &backend=Broker::MEMORY;
event zeek_init()
{
local rec = testrec($a=5);
t["test"] = rec;
rec$a = 6; # This will not propagate to Broker! You have to re-insert.
# Propagate new value to Broker:
t["test"] = rec;
}
&ordered
Used on tables and sets, this attribute ensures that iteration yields members in the order they were inserted. Without this attribute, the iteration order remains undefined. The following is guaranteed to print “foo”, “bar”, and “baz”, in that order:
global sset: set[string] &ordered;
event zeek_init()
{
add sset["foo"];
add sset["bar"];
add sset["baz"];
for ( s in sset )
print s;
}
&deprecated
The associated identifier is marked as deprecated and will be
removed in a future version of Zeek. Look in the NEWS
file for more
instructions to migrate code that uses deprecated functionality.
This attribute can be assigned an optional string literal value to
print along with the deprecation warning. The preferred format of
this warning message should include the version number in which
the identifier will be removed:
type warned: string &deprecated="Remove in vX.Y. This type is deprecated because of reasons, use 'foo' instead.";
&is_assigned
Zeek has static analysis capabilities
for detecting locations in a script that attempt to use a
local variable before it is necessarily defined/assigned. You activate
this using the -u
command-line flag.
However the static analysis lacks sufficient power to tell that some
values are being used safely (guaranteed to have been assigned). In order to
enable users to employ -u
on their own scripts without being
distracted by these false positives, the &is_assigned
attribute can be
associated with a variable to inform Zeek’s analysis that the
script writer asserts the value will be set, suppressing the associated
warnings.
1 event zeek_init()
2 {
3 local a: count;
4 print a;
5 }
$ zeek -b -u test1.zeek
warning in ./test1.zeek, line 4: possibly used without definition (a)
expression error in ./test1.zeek, line 4: value used but not set (a)
1 event zeek_init()
2 {
3 # Note this is not a real place to want to use &is_assigned since it's
4 # clearly a bug, but it demonstrates suppression of warning.
5 local a: count &is_assigned;
6 print a;
7 }
$ zeek -b -u test2.zeek
expression error in ./test2.zeek, line 6: value used but not set (a)
&is_used
Zeek has static analysis capabilities for detecting locations in a script where local variables are assigned values that are not subsequently used (i.e. “dead code”).
It can also warn about unused functions, hooks, and event handlers. The intent behind these checks is to catch instances where the script writer has introduced typos in names, or has forgotten to remove code that’s no longer needed. For functions and hooks, “unused” means the function/hook is neither exported nor in the global scope, and no “live” (i.e., not “unused”) function/hook/event handler calls it. For event handlers, “unused” means that the event engine does not generate the event, nor do any “live” functions/hooks/event handlers generate it.
Zeek never reports any functions/hooks/event handlers that are marked deprecated
(via &deprecated
) as unused.
For cases where it’s desirable to suppress the warning, the
&is_used
attribute may be applied, for example:
1 module Test;
2
3 export {
4 global baz: function();
5 }
6
7 function foo()
8 {
9 }
10
11 function bar() &is_used
12 {
13 }
14
15 function baz()
16 {
17 }
18
19 event zeek_init()
20 {
21 local please_warn: string = "test";
22 local please_no_warning: string = "test" &is_used;
23 }
$ zeek -a -b -u test.zeek
warning in ./test.zeek, line 7: non-exported function does not have any callers (Test::foo)
warning: Test::please_warn assignment unused: Test::please_warn = test; ./test.zeek, line 21
&group
The &group attribute can be used on event handlers and hooks to add them into event groups. By default, all event groups are enabled. Disabling an event group disables all event handlers and hooks with a matching &group attribute. When an event handler or hook is part of multiple groups it is enabled only if all groups are enabled.
event http_request(c: connection, method: string, original_URI: string, unescaped_URI: string, version: string) &group="my-http-group"
{
...
}
event zeek_init()
{
disable_event_group("my-http-group");
}
See also the documentation for the functions enable_event_group
and disable_event_group
.