base/frameworks/logging/main.bro¶
-
Log
¶
The Bro logging interface.
See Logging Framework for an introduction to Bro’s logging framework.
Namespace: | Log |
---|---|
Imports: | base/bif/logging.bif.bro |
Summary¶
Redefinable Options¶
Log::default_ext_prefix : string &redef |
A prefix for extension fields which can be optionally prefixed on all log lines by setting the ext_func field in the log filter. |
Log::default_field_name_map : table &redef |
Default field name mapping for renaming fields in a logging framework filter. |
Log::default_mail_alarms_interval : interval &redef |
Default alarm summary mail interval. |
Log::default_rotation_date_format : string &redef |
Default naming format for timestamps embedded into filenames. |
Log::default_rotation_interval : interval &redef |
Default rotation interval to use for filters that do not specify an interval. |
Log::default_rotation_postprocessor_cmd : string &redef |
Default shell command to run on rotated files. |
Log::default_rotation_postprocessors : table &redef |
Specifies the default postprocessor function per writer type. |
Log::default_scope_sep : string &redef |
Default separator for log field scopes when logs are unrolled and flattened. |
Log::default_writer : Log::Writer &redef |
Default writer to use if a filter does not specify anything else. |
Log::empty_field : string &redef |
Default string to use for empty fields. |
Log::enable_local_logging : bool &redef |
If true, local logging is by default enabled for all filters. |
Log::enable_remote_logging : bool &redef |
If true, remote logging is by default enabled for all filters. |
Log::separator : string &redef |
Default separator to use between fields. |
Log::set_separator : string &redef |
Default separator to use between elements of a set. |
Log::unset_field : string &redef |
Default string to use for an unset &optional field. |
Constants¶
Log::no_filter : Log::Filter |
Sentinel value for indicating that a filter was not found when looked up. |
State Variables¶
Log::active_streams : table |
The streams which are currently active and not disabled. |
Types¶
Log::Filter : record |
A filter type describes how to customize logging streams. |
Log::ID : enum |
Type that defines an ID unique to each log stream. |
Log::RotationInfo : record |
Information passed into rotation callback functions. |
Log::Stream : record |
Type defining the content of a logging stream. |
Log::Writer : enum |
Functions¶
Log::add_default_filter : function |
Adds a default Log::Filter record with name field
set as “default” to a given logging stream. |
Log::add_filter : function |
Adds a custom filter to an existing logging stream. |
Log::create_stream : function |
Creates a new logging stream with the default filter. |
Log::default_ext_func : function &redef |
Default log extension function in the case that you would like to apply the same extensions to all logs. |
Log::default_path_func : function &redef |
Builds the default path values for log filters if not otherwise specified by a filter. |
Log::disable_stream : function |
Disables a currently enabled logging stream. |
Log::enable_stream : function |
Enables a previously disabled logging stream. |
Log::flush : function |
Flushes any currently buffered output for all the writers of a given logging stream. |
Log::get_filter : function |
Gets a filter associated with an existing logging stream. |
Log::get_filter_names : function |
Gets the names of all filters associated with an existing logging stream. |
Log::remove_default_filter : function |
Removes the Log::Filter with name field equal to
“default”. |
Log::remove_filter : function |
Removes a filter from an existing logging stream. |
Log::remove_stream : function |
Removes a logging stream completely, stopping all the threads. |
Log::run_rotation_postprocessor_cmd : function |
Runs a command given by Log::default_rotation_postprocessor_cmd
on a rotated file. |
Log::set_buf : function |
Sets the buffering status for all the writers of a given logging stream. |
Log::write : function |
Writes a new log line/entry to a logging stream. |
Detailed Interface¶
Redefinable Options¶
-
Log::default_ext_prefix
¶ Type: string
Attributes: &redef
Default: "_"
A prefix for extension fields which can be optionally prefixed on all log lines by setting the ext_func field in the log filter.
-
Log::default_field_name_map
¶ Type: table
[string
] ofstring
Attributes: &redef
Default: {}
Default field name mapping for renaming fields in a logging framework filter. This is typically used to ease integration with external data storage and analysis systems.
-
Log::default_mail_alarms_interval
¶ Type: interval
Attributes: &redef
Default: 0 secs
Default alarm summary mail interval. Zero disables alarm summary mails.
Note that this is overridden by the BroControl MailAlarmsInterval option.
-
Log::default_rotation_date_format
¶ Type: string
Attributes: &redef
Default: "%Y-%m-%d-%H-%M-%S"
Default naming format for timestamps embedded into filenames. Uses a
strftime()
style.
-
Log::default_rotation_interval
¶ Type: interval
Attributes: &redef
Default: 0 secs
Default rotation interval to use for filters that do not specify an interval. Zero disables rotation.
Note that this is overridden by the BroControl LogRotationInterval option.
-
Log::default_rotation_postprocessor_cmd
¶ Type: string
Attributes: &redef
Default: ""
Default shell command to run on rotated files. Empty for none.
-
Log::default_rotation_postprocessors
¶ Type: table
[Log::Writer
] offunction
(info:Log::RotationInfo
) :bool
Attributes: &redef
Default: { [Log::WRITER_NONE] = LogNone::default_rotation_postprocessor_func { return (T); }, [Log::WRITER_ASCII] = LogAscii::default_rotation_postprocessor_func { LogAscii::gz = LogAscii::info$fname[-3, (coerce flattenLogAscii::info$fname to int)] == ".gz" ? ".gz" : ""; LogAscii::bls = getenv("BRO_LOG_SUFFIX"); if ("" == LogAscii::bls) LogAscii::bls = "log"; LogAscii::dst = fmt("%s.%s.%s%s", LogAscii::info$path, strftime(Log::default_rotation_date_format, LogAscii::info$open), LogAscii::bls, LogAscii::gz); system(fmt("/bin/mv %s %s", LogAscii::info$fname, LogAscii::dst)); return (Log::run_rotation_postprocessor_cmd(LogAscii::info, LogAscii::dst)); } }
Specifies the default postprocessor function per writer type. Entries in this table are initialized by each writer type.
-
Log::default_scope_sep
¶ Type: string
Attributes: &redef
Default: "."
Default separator for log field scopes when logs are unrolled and flattened. This will be the string between field name components. For example, setting this to “_” will cause the typical field “id.orig_h” to turn into “id_orig_h”.
-
Log::default_writer
¶ Type: Log::Writer
Attributes: &redef
Default: Log::WRITER_ASCII
Default writer to use if a filter does not specify anything else.
-
Log::empty_field
¶ Type: string
Attributes: &redef
Default: "(empty)"
Default string to use for empty fields. This should be different from unset_field to make the output unambiguous. Individual writers can use a different value.
-
Log::enable_local_logging
¶ Type: bool
Attributes: &redef
Default: T
If true, local logging is by default enabled for all filters.
-
Log::enable_remote_logging
¶ Type: bool
Attributes: &redef
Default: T
If true, remote logging is by default enabled for all filters.
-
Log::separator
¶ Type: string
Attributes: &redef
Default: "\x09"
Default separator to use between fields. Individual writers can use a different value.
Constants¶
-
Log::no_filter
¶ Type: Log::Filter
Default: { name="<not found>" writer=Log::WRITER_ASCII pred=<uninitialized> path=<uninitialized> path_func=<uninitialized> include=<uninitialized> exclude=<uninitialized> log_local=T log_remote=T field_name_map={ } scope_sep="." ext_prefix="_" ext_func=anonymous-function ; interv=0 secs postprocessor=<uninitialized> config={ } }
Sentinel value for indicating that a filter was not found when looked up.
State Variables¶
-
Log::active_streams
¶ Type: table
[Log::ID
] ofLog::Stream
Default: {}
The streams which are currently active and not disabled. This table is not meant to be modified by users! Only use it for examining which streams are active.
Types¶
-
Log::Filter
¶ Type: - name:
string
Descriptive name to reference this filter.
- writer:
Log::Writer
&default
=Log::default_writer
&optional
The logging writer implementation to use.
- pred:
function
(rec:any
) :bool
&optional
Indicates whether a log entry should be recorded. If not given, all entries are recorded.
rec: An instance of the stream’s columns
type with its fields set to the values to be logged.returns: True if the entry is to be recorded. - path:
string
&optional
Output path for recording entries matching this filter.
The specific interpretation of the string is up to the logging writer, and may for example be the destination file name. Generally, filenames are expected to be given without any extensions; writers will add appropriate extensions automatically.
If this path is found to conflict with another filter’s for the same writer type, it is automatically corrected by appending “-N”, where N is the smallest integer greater or equal to 2 that allows the corrected path name to not conflict with another filter’s.
- path_func:
function
(id:Log::ID
, path:string
, rec:any
) :string
&optional
A function returning the output path for recording entries matching this filter. This is similar to path yet allows to compute the string dynamically. It is ok to return different strings for separate calls, but be careful: it’s easy to flood the disk by returning a new string for each connection. Upon adding a filter to a stream, if neither
path
norpath_func
is explicitly set by them, thenLog::default_path_func
is used.id: The ID associated with the log stream. path: A suggested path value, which may be either the filter’s path
if defined, else a previous result from the function. If nopath
is defined for the filter, then the first call to the function will contain an empty string.rec: An instance of the stream’s columns
type with its fields set to the values to be logged.returns: The path to be used for the filter, which will be subject to the same automatic correction rules as the path field of Log::Filter
in the case of conflicts with other filters trying to use the same writer/path pair.- include:
set
[string
]&optional
Subset of column names to record. If not given, all columns are recorded.
- exclude:
set
[string
]&optional
Subset of column names to exclude from recording. If not given, all columns are recorded.
- log_local:
bool
&default
=Log::enable_local_logging
&optional
If true, entries are recorded locally.
- log_remote:
bool
&default
=Log::enable_remote_logging
&optional
If true, entries are passed on to remote peers.
- field_name_map:
table
[string
] ofstring
&default
=Log::default_field_name_map
&optional
Field name map to rename fields before the fields are written to the output.
- scope_sep:
string
&default
=Log::default_scope_sep
&optional
A string that is used for unrolling and flattening field names for nested record types.
- ext_prefix:
string
&default
=Log::default_ext_prefix
&optional
Default prefix for all extension fields. It’s typically prudent to set this to something that Bro’s logging framework can’t normally write out in a field name.
- ext_func:
function
(path:string
) :any
&default
=Log::default_ext_func
&optional
Function to collect a log extension value. If not specified, no log extension will be provided for the log. The return value from the function must be a record.
- interv:
interval
&default
=Log::default_rotation_interval
&optional
Rotation interval. Zero disables rotation.
- postprocessor:
function
(info:Log::RotationInfo
) :bool
&optional
Callback function to trigger for rotated files. If not set, the default comes out of
Log::default_rotation_postprocessors
.- config:
table
[string
] ofstring
&default
={ }
&optional
A key/value table that will be passed on to the writer. Interpretation of the values is left to the writer, but usually they will be used for configuration purposes.
A filter type describes how to customize logging streams.
- name:
-
Log::ID
¶ Type: -
Log::UNKNOWN
¶ Dummy place-holder.
-
Broker::LOG
¶ (present if base/frameworks/broker/log.bro is loaded)
-
Files::LOG
¶ (present if base/frameworks/files/main.bro is loaded)
Logging stream for file analysis.
-
Reporter::LOG
¶ (present if base/frameworks/reporter/main.bro is loaded)
-
Cluster::LOG
¶ (present if base/frameworks/cluster/main.bro is loaded)
-
Notice::LOG
¶ (present if base/frameworks/notice/main.bro is loaded)
This is the primary logging stream for notices.
-
Notice::ALARM_LOG
¶ (present if base/frameworks/notice/main.bro is loaded)
This is the alarm stream.
-
Weird::LOG
¶ (present if base/frameworks/notice/weird.bro is loaded)
-
NetControl::LOG
¶ (present if base/frameworks/netcontrol/main.bro is loaded)
-
OpenFlow::LOG
¶ (present if base/frameworks/openflow/plugins/log.bro is loaded)
-
NetControl::DROP
¶ (present if base/frameworks/netcontrol/types.bro is loaded)
Stop forwarding all packets matching the entity.
No additional arguments.
-
NetControl::SHUNT
¶ (present if base/frameworks/netcontrol/shunt.bro is loaded)
-
NetControl::CATCH_RELEASE
¶ (present if base/frameworks/netcontrol/catch-and-release.bro is loaded)
-
DPD::LOG
¶ (present if base/frameworks/dpd/main.bro is loaded)
-
Signatures::LOG
¶ (present if base/frameworks/signatures/main.bro is loaded)
-
PacketFilter::LOG
¶ (present if base/frameworks/packet-filter/main.bro is loaded)
-
Software::LOG
¶ (present if base/frameworks/software/main.bro is loaded)
-
Intel::LOG
¶ (present if base/frameworks/intel/main.bro is loaded)
-
Config::LOG
¶ (present if base/frameworks/config/main.bro is loaded)
-
Tunnel::LOG
¶ (present if base/frameworks/tunnels/main.bro is loaded)
-
Conn::LOG
¶ (present if base/protocols/conn/main.bro is loaded)
-
DCE_RPC::LOG
¶ (present if base/protocols/dce-rpc/main.bro is loaded)
-
DHCP::LOG
¶ (present if base/protocols/dhcp/main.bro is loaded)
-
DNP3::LOG
¶ (present if base/protocols/dnp3/main.bro is loaded)
-
DNS::LOG
¶ (present if base/protocols/dns/main.bro is loaded)
-
FTP::LOG
¶ (present if base/protocols/ftp/main.bro is loaded)
-
SSL::LOG
¶ (present if base/protocols/ssl/main.bro is loaded)
-
X509::LOG
¶ (present if base/files/x509/main.bro is loaded)
-
HTTP::LOG
¶ (present if base/protocols/http/main.bro is loaded)
-
IRC::LOG
¶ (present if base/protocols/irc/main.bro is loaded)
-
KRB::LOG
¶ (present if base/protocols/krb/main.bro is loaded)
-
Modbus::LOG
¶ (present if base/protocols/modbus/main.bro is loaded)
-
mysql::LOG
¶ (present if base/protocols/mysql/main.bro is loaded)
-
NTLM::LOG
¶ (present if base/protocols/ntlm/main.bro is loaded)
-
RADIUS::LOG
¶ (present if base/protocols/radius/main.bro is loaded)
-
RDP::LOG
¶ (present if base/protocols/rdp/main.bro is loaded)
-
RFB::LOG
¶ (present if base/protocols/rfb/main.bro is loaded)
-
SIP::LOG
¶ (present if base/protocols/sip/main.bro is loaded)
-
SNMP::LOG
¶ (present if base/protocols/snmp/main.bro is loaded)
-
SMB::AUTH_LOG
¶ (present if base/protocols/smb/main.bro is loaded)
-
SMB::MAPPING_LOG
¶ (present if base/protocols/smb/main.bro is loaded)
-
SMB::FILES_LOG
¶ (present if base/protocols/smb/main.bro is loaded)
-
SMTP::LOG
¶ (present if base/protocols/smtp/main.bro is loaded)
-
SOCKS::LOG
¶ (present if base/protocols/socks/main.bro is loaded)
-
SSH::LOG
¶ (present if base/protocols/ssh/main.bro is loaded)
-
Syslog::LOG
¶ (present if base/protocols/syslog/main.bro is loaded)
-
PE::LOG
¶ (present if base/files/pe/main.bro is loaded)
-
Unified2::LOG
¶ (present if base/files/unified2/main.bro is loaded)
-
OCSP::LOG
¶ (present if policy/files/x509/log-ocsp.bro is loaded)
-
Barnyard2::LOG
¶ (present if policy/integration/barnyard2/main.bro is loaded)
-
CaptureLoss::LOG
¶ (present if policy/misc/capture-loss.bro is loaded)
-
Traceroute::LOG
¶ (present if policy/misc/detect-traceroute/main.bro is loaded)
-
LoadedScripts::LOG
¶ (present if policy/misc/loaded-scripts.bro is loaded)
-
Stats::LOG
¶ (present if policy/misc/stats.bro is loaded)
-
WeirdStats::LOG
¶ (present if policy/misc/weird-stats.bro is loaded)
-
Known::HOSTS_LOG
¶ (present if policy/protocols/conn/known-hosts.bro is loaded)
-
Known::SERVICES_LOG
¶ (present if policy/protocols/conn/known-services.bro is loaded)
-
Known::MODBUS_LOG
¶ (present if policy/protocols/modbus/known-masters-slaves.bro is loaded)
-
Modbus::REGISTER_CHANGE_LOG
¶ (present if policy/protocols/modbus/track-memmap.bro is loaded)
-
SMB::CMD_LOG
¶ (present if policy/protocols/smb/log-cmds.bro is loaded)
-
Known::CERTS_LOG
¶ (present if policy/protocols/ssl/known-certs.bro is loaded)
-
BroxygenExample::LOG
¶ (present if broxygen/example.bro is loaded)
Type that defines an ID unique to each log stream. Scripts creating new log streams need to redef this enum to add their own specific log ID. The log ID implicitly determines the default name of the generated log file.
-
-
Log::RotationInfo
¶ Type: Information passed into rotation callback functions.
-
Log::Stream
¶ Type: - columns:
any
A record type defining the log’s columns.
- ev:
any
&optional
Event that will be raised once for each log entry. The event receives a single same parameter, an instance of type
columns
.- path:
string
&optional
A path that will be inherited by any filters added to the stream which do not already specify their own path.
Type defining the content of a logging stream.
- columns:
Functions¶
-
Log::add_default_filter
¶ Type: function
(id:Log::ID
) :bool
Adds a default
Log::Filter
record withname
field set as “default” to a given logging stream.Id: The ID associated with a logging stream for which to add a default filter. Returns: The status of a call to Log::add_filter
using a defaultLog::Filter
argument withname
field set to “default”.See also:
Log::add_filter
,Log::remove_filter
,Log::remove_default_filter
-
Log::add_filter
¶ Type: function
(id:Log::ID
, filter:Log::Filter
) :bool
Adds a custom filter to an existing logging stream. If a filter with a matching
name
field already exists for the stream, it is removed when the new filter is successfully added.Id: The ID associated with the logging stream to filter. Filter: A record describing the desired logging parameters. Returns: True if the filter was successfully added, false if the filter was not added or the filter argument was not the correct type. See also:
Log::remove_filter
,Log::add_default_filter
,Log::remove_default_filter
,Log::get_filter
,Log::get_filter_names
-
Log::create_stream
¶ Type: function
(id:Log::ID
, stream:Log::Stream
) :bool
Creates a new logging stream with the default filter.
Id: The ID enum to be associated with the new logging stream. Stream: A record defining the content that the new stream will log. Returns: True if a new logging stream was successfully created and a default filter added to it. See also:
Log::add_default_filter
,Log::remove_default_filter
-
Log::default_ext_func
¶ Type: function
(path:string
) :any
Attributes: &redef
Default log extension function in the case that you would like to apply the same extensions to all logs. The function must return a record with all of the fields to be included in the log. The default function included here does not return a value, which indicates that no extensions are added.
-
Log::default_path_func
¶ Type: function
(id:Log::ID
, path:string
, rec:any
) :string
Attributes: &redef
Builds the default path values for log filters if not otherwise specified by a filter. The default implementation uses id to derive a name. Upon adding a filter to a stream, if neither
path
norpath_func
is explicitly set by them, then this function is used as thepath_func
.Id: The ID associated with the log stream. Path: A suggested path value, which may be either the filter’s path
if defined, else a previous result from the function. If nopath
is defined for the filter, then the first call to the function will contain an empty string.Rec: An instance of the stream’s columns
type with its fields set to the values to be logged.Returns: The path to be used for the filter.
-
Log::disable_stream
¶ Type: function
(id:Log::ID
) :bool
Disables a currently enabled logging stream. Disabled streams will not be written to until they are enabled again. New streams are enabled by default.
Id: The ID associated with the logging stream to disable. Returns: True if the stream is now disabled or was already disabled. See also:
Log::enable_stream
-
Log::enable_stream
¶ Type: function
(id:Log::ID
) :bool
Enables a previously disabled logging stream. Disabled streams will not be written to until they are enabled again. New streams are enabled by default.
Id: The ID associated with the logging stream to enable. Returns: True if the stream is re-enabled or was not previously disabled. See also:
Log::disable_stream
-
Log::flush
¶ Type: function
(id:Log::ID
) :bool
Flushes any currently buffered output for all the writers of a given logging stream.
Id: The ID associated with a logging stream for which to flush buffered data. Returns: True if all writers of a log stream were signalled to flush buffered data or if the logging stream is disabled, false if the logging stream does not exist. See also:
Log::set_buf
,Log::enable_stream
,Log::disable_stream
-
Log::get_filter
¶ Type: function
(id:Log::ID
, name:string
) :Log::Filter
Gets a filter associated with an existing logging stream.
Id: The ID associated with a logging stream from which to obtain one of its filters. Name: A string to match against the name
field of aLog::Filter
for identification purposes.Returns: A filter attached to the logging stream id matching name or, if no matches are found returns the Log::no_filter
sentinel value.See also:
Log::add_filter
,Log::remove_filter
,Log::add_default_filter
,Log::remove_default_filter
,Log::get_filter_names
-
Log::get_filter_names
¶ Type: function
(id:Log::ID
) :set
[string
]Gets the names of all filters associated with an existing logging stream.
Id: The ID of a logging stream from which to obtain the list of filter names. Returns: The set of filter names associated with the stream. - ..bro:see:: Log::remove_filter Log::add_default_filter
- Log::remove_default_filter Log::get_filter
-
Log::remove_default_filter
¶ Type: function
(id:Log::ID
) :bool
Removes the
Log::Filter
withname
field equal to “default”.Id: The ID associated with a logging stream from which to remove the default filter. Returns: The status of a call to Log::remove_filter
using “default” as the argument.See also:
Log::add_filter
,Log::remove_filter
,Log::add_default_filter
-
Log::remove_filter
¶ Type: function
(id:Log::ID
, name:string
) :bool
Removes a filter from an existing logging stream.
Id: The ID associated with the logging stream from which to remove a filter. Name: A string to match against the name
field of aLog::Filter
for identification purposes.Returns: True if the logging stream’s filter was removed or if no filter associated with name was found. See also:
Log::remove_filter
,Log::add_default_filter
,Log::remove_default_filter
,Log::get_filter
,Log::get_filter_names
-
Log::remove_stream
¶ Type: function
(id:Log::ID
) :bool
Removes a logging stream completely, stopping all the threads.
Id: The ID associated with the logging stream. Returns: True if the stream was successfully removed. See also:
Log::create_stream
-
Log::run_rotation_postprocessor_cmd
¶ Type: function
(info:Log::RotationInfo
, npath:string
) :bool
Runs a command given by
Log::default_rotation_postprocessor_cmd
on a rotated file. Meant to be called from postprocessor functions that are added toLog::default_rotation_postprocessors
.Info: A record holding meta-information about the log being rotated. Npath: The new path of the file (after already being rotated/processed by writer-specific postprocessor as defined in Log::default_rotation_postprocessors
).Returns: True when Log::default_rotation_postprocessor_cmd
is empty or the system command given by it has been invoked to postprocess a rotated log file.See also:
Log::default_rotation_date_format
,Log::default_rotation_postprocessor_cmd
,Log::default_rotation_postprocessors
-
Log::set_buf
¶ Type: function
(id:Log::ID
, buffered:bool
) :bool
Sets the buffering status for all the writers of a given logging stream. A given writer implementation may or may not support buffering and if it doesn’t then toggling buffering with this function has no effect.
Id: The ID associated with a logging stream for which to enable/disable buffering. Buffered: Whether to enable or disable log buffering. Returns: True if buffering status was set, false if the logging stream does not exist. See also:
Log::flush
-
Log::write
¶ Type: function
(id:Log::ID
, columns:any
) :bool
Writes a new log line/entry to a logging stream.
Id: The ID associated with a logging stream to be written to. Columns: A record value describing the values of each field/column to write to the log stream. Returns: True if the stream was found and no error occurred in writing to it or if the stream was disabled and nothing was written. False if the stream was not found, or the columns argument did not match what the stream was initially defined to handle, or one of the stream’s filters has an invalid path_func
.See also:
Log::enable_stream
,Log::disable_stream