base/frameworks/logging/main.zeek

Log

The Zeek logging interface.

See Logging Framework for an introduction to Zeek’s logging framework.

Namespace

Log

Imports

base/bif/logging.bif.zeek

Summary

Runtime Options

Log::default_rotation_dir: string &redef

Default rotation directory to use for the dir field of Log::RotationPath during calls to Log::rotation_format_func.

Log::default_rotation_postprocessor_cmd_env: table &redef

This table contains environment variables to be used for the Log::default_rotation_postprocessor_cmd command when executed via Log::run_rotation_postprocessor_cmd.

Redefinable Options

Log::default_ext_prefix: string &redef

A prefix for extension fields which can be optionally prefixed on all log lines by setting the ext_func field in the log filter.

Log::default_field_name_map: table &redef

Default field name mapping for renaming fields in a logging framework filter.

Log::default_logdir: string &redef

Default logging directory.

Log::default_mail_alarms_interval: interval &redef

Default alarm summary mail interval.

Log::default_max_delay_interval: interval &redef

Maximum default log write delay for a stream.

Log::default_max_delay_queue_size: count &redef

The maximum length of the write delay queue per stream.

Log::default_rotation_date_format: string &redef

Default naming format for timestamps embedded into filenames.

Log::default_rotation_interval: interval &redef

Default rotation interval to use for filters that do not specify an interval.

Log::default_rotation_postprocessor_cmd: string &redef

Default shell command to run on rotated files.

Log::default_rotation_postprocessors: table &redef

Specifies the default postprocessor function per writer type.

Log::default_scope_sep: string &redef

Default separator for log field scopes when logs are unrolled and flattened.

Log::default_writer: Log::Writer &redef

Default writer to use if a filter does not specify anything else.

Log::empty_field: string &redef

Default string to use for empty fields.

Log::enable_local_logging: bool &redef

If true, local logging is by default enabled for all filters.

Log::enable_remote_logging: bool &redef

If true, remote logging is by default enabled for all filters.

Log::print_log_path: string &redef

If Log::print_to_log is enabled to write to a print log, this is the path to which the print Log Stream writes to

Log::print_to_log: Log::PrintLogType &redef

Set configuration for print statements redirected to logs.

Log::separator: string &redef

Default separator to use between fields.

Log::set_separator: string &redef

Default separator to use between elements of a set.

Log::unset_field: string &redef

Default string to use for an unset &optional field.

Constants

Log::no_filter: Log::Filter

Sentinel value for indicating that a filter was not found when looked up.

State Variables

Log::active_streams: table

The streams which are currently active and not disabled.

Types

Log::DelayToken: opaque

Type of the opaque value returned by Log::delay.

Log::Filter: record

A filter type describes how to customize logging streams.

Log::ID: enum

Type that defines an ID unique to each log stream.

Log::PolicyHook: hook

A hook type to implement filtering policy at log filter granularity.

Log::PostDelayCallback: function

Type of function to invoke when delaying a log write has completed.

Log::PrintLogInfo: record

If Log::print_to_log is set to redirect, print statements will automatically populate log entries with the fields contained in this record.

Log::PrintLogType: enum

Configurations for Log::print_to_log

Log::RotationFmtInfo: record

Information passed into rotation format callback function given by Log::rotation_format_func.

Log::RotationInfo: record

Information passed into rotation callback functions.

Log::RotationPath: record

A log file rotation path specification that’s returned by the user-customizable Log::rotation_format_func.

Log::RotationPostProcessorFunc: function

The function type for log rotation post processors.

Log::Stream: record

Type defining the content of a logging stream.

Log::StreamPolicyHook: hook

A hook type to implement filtering policy.

Log::Writer: enum

Redefinitions

Log::Filter: record

New Fields

Log::Filter

policy: Log::PolicyHook &optional

Policy hooks can adjust log entry values and veto the writing of a log entry for the record passed into it.

Log::default_rotation_postprocessors: table &redef

Events

Log::log_print: event

Event for accessing logged print records.

Hooks

Log::log_stream_policy: Log::StreamPolicyHook

The global log policy hook.

Functions

Log::add_default_filter: function

Adds a default Log::Filter record with name field set as “default” to a given logging stream.

Log::add_filter: function

Adds a custom filter to an existing logging stream.

Log::create_stream: function

Creates a new logging stream with the default filter.

Log::default_ext_func: function &redef

Default log extension function in the case that you would like to apply the same extensions to all logs.

Log::default_path_func: function &redef

Builds the default path values for log filters if not otherwise specified by a filter.

Log::delay: function

Delay a log write.

Log::delay_finish: function

Release a delay reference taken with Log::delay.

Log::disable_stream: function

Disables a currently enabled logging stream.

Log::empty_post_delay_cb: function

Represents a post delay callback that simply returns T.

Log::enable_stream: function

Enables a previously disabled logging stream.

Log::flush: function

Flushes any currently buffered output for all the writers of a given logging stream.

Log::get_delay_queue_size: function

Get the current size of the delay queue for a stream.

Log::get_filter: function

Gets a filter associated with an existing logging stream.

Log::get_filter_names: function

Gets the names of all filters associated with an existing logging stream.

Log::remove_default_filter: function

Removes the Log::Filter with name field equal to “default”.

Log::remove_filter: function

Removes a filter from an existing logging stream.

Log::remove_stream: function

Removes a logging stream completely, stopping all the threads.

Log::rotation_format_func: function &redef

A function that one may use to customize log file rotation paths.

Log::run_rotation_postprocessor_cmd: function

Runs a command given by Log::default_rotation_postprocessor_cmd on a rotated file.

Log::set_buf: function

Sets the buffering status for all the writers of a given logging stream.

Log::set_max_delay_interval: function

Set the maximum delay for a stream.

Log::set_max_delay_queue_size: function

Set the given stream’s delay queue size.

Log::write: function

Writes a new log line/entry to a logging stream.

Detailed Interface

Runtime Options

Log::default_rotation_dir
Type

string

Attributes

&redef

Default

""

Redefinition

from policy/frameworks/management/persistence.zeek

=:

build_path(Management::get_spool_dir(), log-queue)

Default rotation directory to use for the dir field of Log::RotationPath during calls to Log::rotation_format_func. An empty string implies using the current working directory;

Log::default_rotation_postprocessor_cmd_env
Type

table [string] of string

Attributes

&redef

Default

{}

This table contains environment variables to be used for the Log::default_rotation_postprocessor_cmd command when executed via Log::run_rotation_postprocessor_cmd.

The entries in this table will be prepended with ZEEK_ARG_ as done by system_env.

Redefinable Options

Log::default_ext_prefix
Type

string

Attributes

&redef

Default

"_"

A prefix for extension fields which can be optionally prefixed on all log lines by setting the ext_func field in the log filter.

Log::default_field_name_map
Type

table [string] of string

Attributes

&redef

Default

{}

Default field name mapping for renaming fields in a logging framework filter. This is typically used to ease integration with external data storage and analysis systems.

Log::default_logdir
Type

string

Attributes

&redef

Default

""

Default logging directory. An empty string implies using the current working directory.

This directory is also used for rotated logs in cases where Log::rotation_format_func returns a record with an empty or unset dir field.

Log::default_mail_alarms_interval
Type

interval

Attributes

&redef

Default

0 secs

Default alarm summary mail interval. Zero disables alarm summary mails.

Note that this is overridden by the ZeekControl MailAlarmsInterval option.

Log::default_max_delay_interval
Type

interval

Attributes

&redef

Default

200.0 msecs

Maximum default log write delay for a stream. A Log::write operation is delayed by at most this interval if Log::delay is called within Log::log_stream_policy.

Log::default_max_delay_queue_size
Type

count

Attributes

&redef

Default

1000

The maximum length of the write delay queue per stream. If exceeded, an attempt is made to evict the oldest writes from the queue. If post delay callbacks re-delay a write operation, the maximum queue size may be exceeded.

Log::default_rotation_date_format
Type

string

Attributes

&redef

Default

"%Y-%m-%d-%H-%M-%S"

Default naming format for timestamps embedded into filenames. Uses a strftime() style.

Log::default_rotation_interval
Type

interval

Attributes

&redef

Default

0 secs

Default rotation interval to use for filters that do not specify an interval. Zero disables rotation.

Note that this is overridden by the ZeekControl LogRotationInterval option.

Log::default_rotation_postprocessor_cmd
Type

string

Attributes

&redef

Default

""

Default shell command to run on rotated files. Empty for none.

Log::default_rotation_postprocessors
Type

table [Log::Writer] of function (info: Log::RotationInfo) : bool

Attributes

&redef

Default

{}

Redefinition

from base/frameworks/logging/main.zeek

+=:

Log::WRITER_ASCII = Log::default_ascii_rotation_postprocessor_func
Redefinition

from base/frameworks/logging/writers/none.zeek

+=:

Log::WRITER_NONE = LogNone::default_rotation_postprocessor_func

Specifies the default postprocessor function per writer type. Entries in this table are initialized by each writer type.

Log::default_scope_sep
Type

string

Attributes

&redef

Default

"."

Default separator for log field scopes when logs are unrolled and flattened. This will be the string between field name components. For example, setting this to “_” will cause the typical field “id.orig_h” to turn into “id_orig_h”.

Log::default_writer
Type

Log::Writer

Attributes

&redef

Default

Log::WRITER_ASCII

Default writer to use if a filter does not specify anything else.

Log::empty_field
Type

string

Attributes

&redef

Default

"(empty)"

Default string to use for empty fields. This should be different from unset_field to make the output unambiguous. Individual writers can use a different value.

Log::enable_local_logging
Type

bool

Attributes

&redef

Default

T

If true, local logging is by default enabled for all filters.

Log::enable_remote_logging
Type

bool

Attributes

&redef

Default

T

If true, remote logging is by default enabled for all filters.

Log::print_log_path
Type

string

Attributes

&redef

Default

"print"

If Log::print_to_log is enabled to write to a print log, this is the path to which the print Log Stream writes to

Log::print_to_log
Type

Log::PrintLogType

Attributes

&redef

Default

Log::REDIRECT_NONE

Set configuration for print statements redirected to logs.

Log::separator
Type

string

Attributes

&redef

Default

"\x09"

Default separator to use between fields. Individual writers can use a different value.

Log::set_separator
Type

string

Attributes

&redef

Default

","

Default separator to use between elements of a set. Individual writers can use a different value.

Log::unset_field
Type

string

Attributes

&redef

Default

"-"

Default string to use for an unset &optional field. Individual writers can use a different value.

Constants

Log::no_filter
Type

Log::Filter

Default
{
   name="<not found>"
   writer=Log::WRITER_ASCII
   path=<uninitialized>
   path_func=<uninitialized>
   include=<uninitialized>
   exclude=<uninitialized>
   log_local=T
   log_remote=T
   field_name_map={

   }
   scope_sep="."
   ext_prefix="_"
   ext_func=lambda_<2528247166937952945>
   ;
   interv=0 secs
   postprocessor=<uninitialized>
   config={

   }
   policy=<uninitialized>
}

Sentinel value for indicating that a filter was not found when looked up.

State Variables

Log::active_streams
Type

table [Log::ID] of Log::Stream

Default

{}

The streams which are currently active and not disabled. This table is not meant to be modified by users! Only use it for examining which streams are active.

Types

Log::DelayToken
Type

opaque of LogDelayToken

Type of the opaque value returned by Log::delay. These values can be passed to Log::delay_finish to release a delayed write operation.

Log::Filter
Type

record

name: string

Descriptive name to reference this filter.

writer: Log::Writer &default = Log::default_writer &optional

The logging writer implementation to use.

path: string &optional

Output path for recording entries matching this filter.

The specific interpretation of the string is up to the logging writer, and may for example be the destination file name. Generally, filenames are expected to be given without any extensions; writers will add appropriate extensions automatically.

If this path is found to conflict with another filter’s for the same writer type, it is automatically corrected by appending “-N”, where N is the smallest integer greater or equal to 2 that allows the corrected path name to not conflict with another filter’s.

path_func: function (id: Log::ID, path: string, rec: any)string &optional

A function returning the output path for recording entries matching this filter. This is similar to path yet allows to compute the string dynamically. It is ok to return different strings for separate calls, but be careful: it’s easy to flood the disk by returning a new string for each connection. Upon adding a filter to a stream, if neither path nor path_func is explicitly set by them, then Log::default_path_func is used.

param id

The ID associated with the log stream.

param path

A suggested path value, which may be either the filter’s path if defined, else a previous result from the function. If no path is defined for the filter, then the first call to the function will contain an empty string.

param rec

An instance of the stream’s columns type with its fields set to the values to be logged.

returns

The path to be used for the filter, which will be subject to the same automatic correction rules as the path field of Log::Filter in the case of conflicts with other filters trying to use the same writer/path pair.

include: set [string] &optional

Subset of column names to record. If not given, all columns are recorded.

exclude: set [string] &optional

Subset of column names to exclude from recording. If not given, all columns are recorded.

log_local: bool &default = Log::enable_local_logging &optional

If true, entries are recorded locally.

log_remote: bool &default = Log::enable_remote_logging &optional

If true, entries are passed on to remote peers.

field_name_map: table [string] of string &default = Log::default_field_name_map &optional

Field name map to rename fields before the fields are written to the output.

scope_sep: string &default = Log::default_scope_sep &optional

A string that is used for unrolling and flattening field names for nested record types.

ext_prefix: string &default = Log::default_ext_prefix &optional

Default prefix for all extension fields. It’s typically prudent to set this to something that Zeek’s logging framework can’t normally write out in a field name.

ext_func: function (path: string)any &default = Log::default_ext_func &optional

Function to collect a log extension value. If not specified, no log extension will be provided for the log. The return value from the function must be a record.

interv: interval &default = Log::default_rotation_interval &optional

Rotation interval. Zero disables rotation.

postprocessor: function (info: Log::RotationInfo)bool &optional

Callback function to trigger for rotated files. If not set, the default comes out of Log::default_rotation_postprocessors.

config: table [string] of string &default = {  } &optional

A key/value table that will be passed on to the writer. Interpretation of the values is left to the writer, but usually they will be used for configuration purposes.

policy: Log::PolicyHook &optional

Policy hooks can adjust log entry values and veto the writing of a log entry for the record passed into it. Any hook that breaks from its body signals that Zeek won’t log the entry passed into it.

When no policy hook is defined, the filter inherits the hook from the stream it’s associated with.

A filter type describes how to customize logging streams.

Log::ID
Type

enum

Log::UNKNOWN

Dummy place-holder.

Log::PRINTLOG

Print statements that have been redirected to a log stream.

Broker::LOG

(present if base/frameworks/broker/log.zeek is loaded)

Cluster::LOG

(present if base/frameworks/cluster/main.zeek is loaded)

Config::LOG

(present if base/frameworks/config/main.zeek is loaded)

DPD::LOG

(present if base/frameworks/analyzer/dpd.zeek is loaded)

Analyzer::Logging::LOG

(present if base/frameworks/analyzer/logging.zeek is loaded)

Files::LOG

(present if base/frameworks/files/main.zeek is loaded)

Logging stream for file analysis.

Reporter::LOG

(present if base/frameworks/reporter/main.zeek is loaded)

Notice::LOG

(present if base/frameworks/notice/main.zeek is loaded)

This is the primary logging stream for notices.

Notice::ALARM_LOG

(present if base/frameworks/notice/main.zeek is loaded)

This is the alarm stream.

Weird::LOG

(present if base/frameworks/notice/weird.zeek is loaded)

Signatures::LOG

(present if base/frameworks/signatures/main.zeek is loaded)

PacketFilter::LOG

(present if base/frameworks/packet-filter/main.zeek is loaded)

Software::LOG

(present if base/frameworks/software/main.zeek is loaded)

Intel::LOG

(present if base/frameworks/intel/main.zeek is loaded)

Tunnel::LOG

(present if base/frameworks/tunnels/main.zeek is loaded)

OpenFlow::LOG

(present if base/frameworks/openflow/plugins/log.zeek is loaded)

NetControl::LOG

(present if base/frameworks/netcontrol/main.zeek is loaded)

NetControl::DROP_LOG

(present if base/frameworks/netcontrol/drop.zeek is loaded)

NetControl::SHUNT

(present if base/frameworks/netcontrol/shunt.zeek is loaded)

Conn::LOG

(present if base/protocols/conn/main.zeek is loaded)

DCE_RPC::LOG

(present if base/protocols/dce-rpc/main.zeek is loaded)

DHCP::LOG

(present if base/protocols/dhcp/main.zeek is loaded)

DNP3::LOG

(present if base/protocols/dnp3/main.zeek is loaded)

DNS::LOG

(present if base/protocols/dns/main.zeek is loaded)

FTP::LOG

(present if base/protocols/ftp/main.zeek is loaded)

SSL::LOG

(present if base/protocols/ssl/main.zeek is loaded)

X509::LOG

(present if base/files/x509/main.zeek is loaded)

OCSP::LOG

(present if base/files/x509/log-ocsp.zeek is loaded)

HTTP::LOG

(present if base/protocols/http/main.zeek is loaded)

IRC::LOG

(present if base/protocols/irc/main.zeek is loaded)

KRB::LOG

(present if base/protocols/krb/main.zeek is loaded)

LDAP::LDAP_LOG

(present if base/protocols/ldap/main.zeek is loaded)

LDAP::LDAP_SEARCH_LOG

(present if base/protocols/ldap/main.zeek is loaded)

Modbus::LOG

(present if base/protocols/modbus/main.zeek is loaded)

MQTT::CONNECT_LOG

(present if base/protocols/mqtt/main.zeek is loaded)

MQTT::SUBSCRIBE_LOG

(present if base/protocols/mqtt/main.zeek is loaded)

MQTT::PUBLISH_LOG

(present if base/protocols/mqtt/main.zeek is loaded)

mysql::LOG

(present if base/protocols/mysql/main.zeek is loaded)

NTLM::LOG

(present if base/protocols/ntlm/main.zeek is loaded)

NTP::LOG

(present if base/protocols/ntp/main.zeek is loaded)

QUIC::LOG

(present if base/protocols/quic/main.zeek is loaded)

RADIUS::LOG

(present if base/protocols/radius/main.zeek is loaded)

RDP::LOG

(present if base/protocols/rdp/main.zeek is loaded)

RFB::LOG

(present if base/protocols/rfb/main.zeek is loaded)

SIP::LOG

(present if base/protocols/sip/main.zeek is loaded)

SNMP::LOG

(present if base/protocols/snmp/main.zeek is loaded)

SMB::MAPPING_LOG

(present if base/protocols/smb/main.zeek is loaded)

SMB::FILES_LOG

(present if base/protocols/smb/main.zeek is loaded)

SMTP::LOG

(present if base/protocols/smtp/main.zeek is loaded)

SOCKS::LOG

(present if base/protocols/socks/main.zeek is loaded)

SSH::LOG

(present if base/protocols/ssh/main.zeek is loaded)

Syslog::LOG

(present if base/protocols/syslog/main.zeek is loaded)

WebSocket::LOG

(present if base/protocols/websocket/main.zeek is loaded)

PE::LOG

(present if base/files/pe/main.zeek is loaded)

Management::Log::LOG

(present if policy/frameworks/management/log.zeek is loaded)

NetControl::CATCH_RELEASE

(present if policy/frameworks/netcontrol/catch-and-release.zeek is loaded)

Telemetry::LOG

(present if policy/frameworks/telemetry/log.zeek is loaded)

Telemetry::LOG_HISTOGRAM

(present if policy/frameworks/telemetry/log.zeek is loaded)

CaptureLoss::LOG

(present if policy/misc/capture-loss.zeek is loaded)

Traceroute::LOG

(present if policy/misc/detect-traceroute/main.zeek is loaded)

LoadedScripts::LOG

(present if policy/misc/loaded-scripts.zeek is loaded)

Stats::LOG

(present if policy/misc/stats.zeek is loaded)

WeirdStats::LOG

(present if policy/misc/weird-stats.zeek is loaded)

UnknownProtocol::LOG

(present if policy/misc/unknown-protocols.zeek is loaded)

Known::HOSTS_LOG

(present if policy/protocols/conn/known-hosts.zeek is loaded)

Known::SERVICES_LOG

(present if policy/protocols/conn/known-services.zeek is loaded)

Known::MODBUS_LOG

(present if policy/protocols/modbus/known-masters-slaves.zeek is loaded)

Modbus::REGISTER_CHANGE_LOG

(present if policy/protocols/modbus/track-memmap.zeek is loaded)

SMB::CMD_LOG

(present if policy/protocols/smb/log-cmds.zeek is loaded)

Known::CERTS_LOG

(present if policy/protocols/ssl/known-certs.zeek is loaded)

ZeekygenExample::LOG

(present if zeekygen/example.zeek is loaded)

Type that defines an ID unique to each log stream. Scripts creating new log streams need to redef this enum to add their own specific log ID. The log ID implicitly determines the default name of the generated log file.

Log::PolicyHook
Type

hook (rec: any, id: Log::ID, filter: Log::Filter) : bool

A hook type to implement filtering policy at log filter granularity. Like Log::StreamPolicyHook, these can implement added functionality, alter it prior to logging, or veto the write. These hooks run at log filter granularity, so get a Log::Filter instance as additional argument. You can pass additional state into the hook via the the filter$config table.

Param rec

An instance of the stream’s columns type with its fields set to the values to be logged.

Param id

The ID associated with the logging stream the filter belongs to.

Param filter

The Log::Filter instance that steers the output of the given log record.

Log::PostDelayCallback
Type

function (rec: any, id: Log::ID) : bool

Type of function to invoke when delaying a log write has completed.

Functions of this type take the same arguments as Log::StreamPolicyHook and act as a callback passed to zeek:see:Log::delay. They execute just before the record is forwarded to the individual log filters.

Returning F from a post delay callback discards the log write.

Log::PrintLogInfo
Type

record

ts: time &log

The network time at which the print statement was executed.

vals: string_vec &log

Set of strings passed to the print statement.

If Log::print_to_log is set to redirect, print statements will automatically populate log entries with the fields contained in this record.

Log::PrintLogType
Type

enum

Log::REDIRECT_NONE

No redirection of print statements.

Log::REDIRECT_STDOUT

Redirection of those print statements that were being logged to stdout, leaving behind those set to go to other specific files.

Log::REDIRECT_ALL

Redirection of all print statements.

Configurations for Log::print_to_log

Log::RotationFmtInfo
Type

record

writer: Log::Writer

The log writer being used.

path: string

Original path value.

open: time

Time when opened.

close: time

Time when closed.

terminating: bool

True if rotation occurred due to Zeek shutting down.

postprocessor: Log::RotationPostProcessorFunc &optional

The postprocessor function that will be called after rotation.

Information passed into rotation format callback function given by Log::rotation_format_func.

Log::RotationInfo
Type

record

writer: Log::Writer

The log writer being used.

fname: string

Full name of the rotated file.

path: string

Original path value.

open: time

Time when opened.

close: time

Time when closed.

terminating: bool

True if rotation occurred due to Zeek shutting down.

Information passed into rotation callback functions.

Log::RotationPath
Type

record

dir: string &default = Log::default_rotation_dir &optional

A directory to rotate the log to. This directory is created just-in-time, as the log rotation is about to happen. If it cannot be created, an error is emitted and the rotation process tries to proceed with rotation inside the working directory. When setting this field, beware that renaming files across file systems will generally fail.

file_basename: string

A base name to use for the rotated log. Log writers may later append a file extension of their choosing to this user-chosen base (e.g. if using the default ASCII writer and you want rotated files of the format “foo-<date>.log”, then this basename can be set to “foo-<date>” and the “.log” is added later (there’s also generally means of customizing the file extension, too, like the ZEEK_LOG_SUFFIX environment variable or writer-dependent configuration options.

A log file rotation path specification that’s returned by the user-customizable Log::rotation_format_func.

Log::RotationPostProcessorFunc
Type

function (info: Log::RotationInfo) : bool

The function type for log rotation post processors.

Log::Stream
Type

record

columns: any

A record type defining the log’s columns.

ev: any &optional

Event that will be raised once for each log entry. The event receives a single same parameter, an instance of type columns.

path: string &optional

A path that will be inherited by any filters added to the stream which do not already specify their own path.

policy: Log::PolicyHook &optional

Policy hooks can adjust log records and veto their writing. Any hook handler that breaks from its body signals that Zeek won’t log the entry passed into it. You can pass arbitrary state into the hook via the filter instance and its config table.

New Filters created for this stream will inherit this policy hook, unless they provide their own.

event_groups: set [string] &default = {  } &optional

Event groups associated with this stream that are disabled when Log::disable_stream is invoked and re-enabled during Log::enable_stream.

This field can be used to short-circuit event handlers that are solely responsible for logging functionality at runtime when a log stream is disabled.

This field allows for both, attribute event groups and module event groups. If the given group names exists as attribute or module or either event group, they are disabled when the log stream is disabled and enabled when the stream is enabled again.

max_delay_interval: interval &default = Log::default_max_delay_interval &optional

Maximum delay interval for this stream.

This value can be increased using Log::set_max_delay_interval after the stream has been created.

max_delay_queue_size: count &default = Log::default_max_delay_queue_size &optional

Maximum delay queue size of this stream.

This value can be changed using Log::set_max_delay_queue_size after the stream has been created.

Type defining the content of a logging stream.

Log::StreamPolicyHook
Type

hook (rec: any, id: Log::ID) : bool

A hook type to implement filtering policy. Hook handlers run on each log record. They can implement arbitrary per-record processing, alter the log record, or veto the writing of the given record by breaking from the hook handler.

Param rec

An instance of the stream’s columns type with its fields set to the values to be logged.

Param id

The ID associated with the logging stream the filter belongs to.

Log::Writer
Type

enum

Log::WRITER_ASCII
Log::WRITER_NONE
Log::WRITER_SQLITE

Events

Log::log_print
Type

event (rec: Log::PrintLogInfo)

Event for accessing logged print records.

Hooks

Log::log_stream_policy
Type

Log::StreamPolicyHook

The global log policy hook. The framework invokes this hook for any log write, prior to iterating over the stream’s associated filters. As with filter-specific hooks, breaking from the hook vetoes writing of the given log record. Note that filter-level policy hooks still get invoked after the global hook vetoes, but they cannot “un-veto” the write.

Functions

Log::add_default_filter
Type

function (id: Log::ID) : bool

Adds a default Log::Filter record with name field set as “default” to a given logging stream.

Parameters

id – The ID associated with a logging stream for which to add a default filter.

Returns

The status of a call to Log::add_filter using a default Log::Filter argument with name field set to “default”.

See also: Log::add_filter, Log::remove_filter, Log::remove_default_filter

Log::add_filter
Type

function (id: Log::ID, filter: Log::Filter) : bool

Adds a custom filter to an existing logging stream. If a filter with a matching name field already exists for the stream, it is removed when the new filter is successfully added.

Parameters
  • id – The ID associated with the logging stream to filter.

  • filter – A record describing the desired logging parameters.

Returns

True if the filter was successfully added, false if the filter was not added or the filter argument was not the correct type.

See also: Log::remove_filter, Log::add_default_filter, Log::remove_default_filter, Log::get_filter, Log::get_filter_names

Log::create_stream
Type

function (id: Log::ID, stream: Log::Stream) : bool

Creates a new logging stream with the default filter.

Parameters
  • id – The ID enum to be associated with the new logging stream.

  • stream – A record defining the content that the new stream will log.

Returns

True if a new logging stream was successfully created and a default filter added to it.

See also: Log::add_default_filter, Log::remove_default_filter

Log::default_ext_func
Type

function (path: string) : any

Attributes

&redef

Default log extension function in the case that you would like to apply the same extensions to all logs. The function must return a record with all of the fields to be included in the log. The default function included here does not return a value, which indicates that no extensions are added.

Log::default_path_func
Type

function (id: Log::ID, path: string, rec: any) : string

Attributes

&redef

Builds the default path values for log filters if not otherwise specified by a filter. The default implementation uses id to derive a name. Upon adding a filter to a stream, if neither path nor path_func is explicitly set by them, then this function is used as the path_func.

Parameters
  • id – The ID associated with the log stream.

  • path – A suggested path value, which may be either the filter’s path if defined, else a previous result from the function. If no path is defined for the filter, then the first call to the function will contain an empty string.

  • rec – An instance of the stream’s columns type with its fields set to the values to be logged.

Returns

The path to be used for the filter.

Log::delay
Type

function (id: Log::ID, rec: any, post_delay_cb: Log::PostDelayCallback &default = Log::empty_post_delay_cb &optional) : Log::DelayToken

Delay a log write.

Calling this function is currently only allowed within the execution of a Log::log_stream_policy hook and requires the caller to provide the stream ID and log record of the active write operation as parameters.

Conceptually, the delay is inserted between the execution of the

:param zeek:see:Log::log_stream_policy hook and the policy hooks of filters.

Calling this function increments a reference count that can subsequently be decremented using Log::delay_finish. The delay completes when either the reference count reaches zero, or the configured maximum delay interval for the stream expires. The optional post_delay_cb is invoked when the delay completed.

The post_delay_cb function can extend the delay by invoking Log::delay again. There’s no limit to how often a write can be re-delayed. Further, it can discard the log record altogether by returning F. If post_delay_cb is not provided, the behavior is equivalent to a no-op callback solely returning T.

Parameters
  • id – The ID associated with a logging stream.

  • rec – The log record.

  • post_delay_cb – A callback to invoke when the delay completed.

Returns

An opaque token of type Log::DelayToken to be passed to Log::delay_finish.

Log::delay_finish
Type

function (id: Log::ID, rec: any, token: Log::DelayToken) : bool

Release a delay reference taken with Log::delay.

When the last reference is released, Log::delay_finish synchronously resumes the delayed Log::write operation.

Parameters
  • id – The ID associated with a logging stream.

  • rec – The log record.

  • token – The opaque token as returned by Log::delay.

Returns

T on success, F if an inconsistent combination of id, rec and token was provided.

Log::disable_stream
Type

function (id: Log::ID) : bool

Disables a currently enabled logging stream. Disabled streams will not be written to until they are enabled again. New streams are enabled by default.

Parameters

id – The ID associated with the logging stream to disable.

Returns

True if the stream is now disabled or was already disabled.

See also: Log::enable_stream

Log::empty_post_delay_cb
Type

function (rec: any, id: Log::ID) : bool

Represents a post delay callback that simply returns T. This is used as a default value for Log::delay and ignored internally.

Log::enable_stream
Type

function (id: Log::ID) : bool

Enables a previously disabled logging stream. Disabled streams will not be written to until they are enabled again. New streams are enabled by default.

Parameters

id – The ID associated with the logging stream to enable.

Returns

True if the stream is re-enabled or was not previously disabled.

See also: Log::disable_stream

Log::flush
Type

function (id: Log::ID) : bool

Flushes any currently buffered output for all the writers of a given logging stream.

Parameters

id – The ID associated with a logging stream for which to flush buffered data.

Returns

True if all writers of a log stream were signalled to flush buffered data or if the logging stream is disabled, false if the logging stream does not exist.

See also: Log::set_buf, Log::enable_stream, Log::disable_stream

Log::get_delay_queue_size
Type

function (id: Log::ID) : int

Get the current size of the delay queue for a stream.

Parameters

id – The ID associated with a logging stream.

Returns

The current size of the delay queue, or -1 on error.

Log::get_filter
Type

function (id: Log::ID, name: string) : Log::Filter

Gets a filter associated with an existing logging stream.

Parameters
  • id – The ID associated with a logging stream from which to obtain one of its filters.

  • name – A string to match against the name field of a Log::Filter for identification purposes.

Returns

A filter attached to the logging stream id matching name or, if no matches are found returns the Log::no_filter sentinel value.

See also: Log::add_filter, Log::remove_filter, Log::add_default_filter, Log::remove_default_filter, Log::get_filter_names

Log::get_filter_names
Type

function (id: Log::ID) : set [string]

Gets the names of all filters associated with an existing logging stream.

Parameters

id – The ID of a logging stream from which to obtain the list of filter names.

Returns

The set of filter names associated with the stream.

..zeek:see:: Log::remove_filter Log::add_default_filter

Log::remove_default_filter Log::get_filter

Log::remove_default_filter
Type

function (id: Log::ID) : bool

Removes the Log::Filter with name field equal to “default”.

Parameters

id – The ID associated with a logging stream from which to remove the default filter.

Returns

The status of a call to Log::remove_filter using “default” as the argument.

See also: Log::add_filter, Log::remove_filter, Log::add_default_filter

Log::remove_filter
Type

function (id: Log::ID, name: string) : bool

Removes a filter from an existing logging stream.

Parameters
  • id – The ID associated with the logging stream from which to remove a filter.

  • name – A string to match against the name field of a Log::Filter for identification purposes.

Returns

True if the logging stream’s filter was removed or if no filter associated with name was found.

See also: Log::remove_filter, Log::add_default_filter, Log::remove_default_filter, Log::get_filter, Log::get_filter_names

Log::remove_stream
Type

function (id: Log::ID) : bool

Removes a logging stream completely, stopping all the threads.

Parameters

id – The ID associated with the logging stream.

Returns

True if the stream was successfully removed.

See also: Log::create_stream

Log::rotation_format_func
Type

function (ri: Log::RotationFmtInfo) : Log::RotationPath

Attributes

&redef

A function that one may use to customize log file rotation paths.

Log::run_rotation_postprocessor_cmd
Type

function (info: Log::RotationInfo, npath: string) : bool

Runs a command given by Log::default_rotation_postprocessor_cmd on a rotated file. Meant to be called from postprocessor functions that are added to Log::default_rotation_postprocessors.

Parameters
  • info – A record holding meta-information about the log being rotated.

  • npath – The new path of the file (after already being rotated/processed by writer-specific postprocessor as defined in Log::default_rotation_postprocessors).

Returns

True when Log::default_rotation_postprocessor_cmd is empty or the system command given by it has been invoked to postprocess a rotated log file.

See also: Log::default_rotation_date_format, Log::default_rotation_postprocessor_cmd_env, Log::default_rotation_postprocessor_cmd, Log::default_rotation_postprocessors

Log::set_buf
Type

function (id: Log::ID, buffered: bool) : bool

Sets the buffering status for all the writers of a given logging stream. A given writer implementation may or may not support buffering and if it doesn’t then toggling buffering with this function has no effect.

Parameters
  • id – The ID associated with a logging stream for which to enable/disable buffering.

  • buffered – Whether to enable or disable log buffering.

Returns

True if buffering status was set, false if the logging stream does not exist.

See also: Log::flush

Log::set_max_delay_interval
Type

function (id: Log::ID, max_delay: interval) : bool

Set the maximum delay for a stream.

Multiple calls to this function will only ever increase the maximum delay, the delay cannot be lowered. The default maximum delay for a stream is zeek:see:Log::default_max_delay_interval.

When a stream is removed and re-created via Log::create_stream, the new stream is re-configured with the previously used maximum delay.

Parameters
  • id – The ID associated with a logging stream.

  • max_delay – The maximum delay interval for this stream.

Returns

T on success, else F.

Log::set_max_delay_queue_size
Type

function (id: Log::ID, max_size: count) : bool

Set the given stream’s delay queue size.

If the queue holds more records than the given queue_size, these are attempted to be evicted at the time of the call.

When a stream is removed and re-created via Log::create_stream, the new stream is re-configured with the most recently used queue size.

Parameters
  • id – The ID associated with a logging stream.

  • max_delay – The maximum delay interval of this stream.

Returns

T on success, else F.

Log::write
Type

function (id: Log::ID, columns: any) : bool

Writes a new log line/entry to a logging stream.

Parameters
  • id – The ID associated with a logging stream to be written to.

  • columns – A record value describing the values of each field/column to write to the log stream.

Returns

True if the stream was found and no error occurred in writing to it or if the stream was disabled and nothing was written. False if the stream was not found, or the columns argument did not match what the stream was initially defined to handle, or one of the stream’s filters has an invalid path_func.

See also: Log::enable_stream, Log::disable_stream