base/frameworks/input/main.zeek

Input

The input framework provides a way to read previously stored data either as an event stream or into a Zeek table.

Namespace

Input

Imports

base/bif/input.bif.zeek

Summary

Runtime Options

Input::default_mode: Input::Mode &redef

The default reader mode used.

Input::default_reader: Input::Reader &redef

The default input reader used.

Redefinable Options

Input::accept_unsupported_types: bool &redef

Flag that controls if the input framework accepts records that contain types that are not supported (at the moment file and function).

Input::empty_field: string &redef

String to use for empty fields.

Input::separator: string &redef

Separator between fields.

Input::set_separator: string &redef

Separator between set elements.

Input::unset_field: string &redef

String to use for an unset &optional field.

Types

Input::AnalysisDescription: record

A file analysis input stream type used to forward input data to the file analysis framework.

Input::Event: enum

Type that describes what kind of change occurred.

Input::EventDescription: record

An event input stream type used to send input data to a Zeek event.

Input::Mode: enum

Type that defines the input stream read mode.

Input::TableDescription: record

A table input stream type used to send data to a Zeek table.

Input::Reader: enum

Events

Input::end_of_data: event

Event that is called when the end of a data source has been reached, including after an update.

Functions

Input::add_analysis: function

Create a new file analysis input stream from a given source.

Input::add_event: function

Create a new event input stream from a given source.

Input::add_table: function

Create a new table input stream from a given source.

Input::force_update: function

Forces the current input to be checked for changes.

Input::remove: function

Remove an input stream.

Detailed Interface

Runtime Options

Input::default_mode
Type

Input::Mode

Attributes

&redef

Default

Input::MANUAL

The default reader mode used. Defaults to MANUAL.

Input::default_reader
Type

Input::Reader

Attributes

&redef

Default

Input::READER_ASCII

The default input reader used. Defaults to READER_ASCII.

Redefinable Options

Input::accept_unsupported_types
Type

bool

Attributes

&redef

Default

F

Flag that controls if the input framework accepts records that contain types that are not supported (at the moment file and function). If true, the input framework will warn in these cases, but continue. If false, it will abort. Defaults to false (abort).

Input::empty_field
Type

string

Attributes

&redef

Default

"(empty)"

String to use for empty fields. Individual readers can use a different value.

Input::separator
Type

string

Attributes

&redef

Default

"\x09"

Separator between fields. Please note that the separator has to be exactly one character long. Individual readers can use a different value.

Input::set_separator
Type

string

Attributes

&redef

Default

","

Separator between set elements. Please note that the separator has to be exactly one character long. Individual readers can use a different value.

Input::unset_field
Type

string

Attributes

&redef

Default

"-"

String to use for an unset &optional field. Individual readers can use a different value.

Types

Input::AnalysisDescription
Type

record

source: string

String that allows the reader to find the source. For READER_ASCII, this is the filename.

reader: Input::Reader &default = Input::READER_BINARY &optional

Reader to use for this stream. Compatible readers must be able to accept a filter of a single string type (i.e. they read a byte stream).

mode: Input::Mode &default = Input::default_mode &optional

Read mode to use for this stream.

name: string

Descriptive name that uniquely identifies the input source. Can be used to remove a stream at a later time. This will also be used for the unique source field of fa_file. Most of the time, the best choice for this field will be the same value as the source field.

config: table [string] of string &default = {  } &optional

A key/value table that will be passed to the reader. Interpretation of the values is left to the reader, but usually they will be used for configuration purposes.

A file analysis input stream type used to forward input data to the file analysis framework.

Input::Event
Type

enum

Input::EVENT_NEW

New data has been imported.

Input::EVENT_CHANGED

Existing data has been changed.

Input::EVENT_REMOVED

Previously existing data has been removed.

Type that describes what kind of change occurred.

Input::EventDescription
Type

record

source: string

String that allows the reader to find the source. For READER_ASCII, this is the filename.

reader: Input::Reader &default = Input::default_reader &optional

Reader to use for this stream.

mode: Input::Mode &default = Input::default_mode &optional

Read mode to use for this stream.

name: string

Descriptive name. Used to remove a stream at a later time.

fields: any

Record type describing the fields to be retrieved from the input source.

want_record: bool &default = T &optional

If this is false, the event receives each value in fields as a separate argument. If this is set to true (default), the event receives all fields in a single record value.

ev: any

The event that is raised each time a new line is received from the reader. The event will receive an Input::EventDescription record as the first argument, an Input::Event enum as the second argument, and the fields (as specified in fields) as the following arguments (this will either be a single record value containing all fields, or each field value as a separate argument).

error_ev: any &optional

Error event that is raised when an information, warning or error is raised by the input stream. If the level is error, the stream will automatically be closed. The event receives the Input::EventDescription as the first argument, the message as the second argument and the Reporter::Level as the third argument.

The event is raised like it had been declared as follows: error_ev: function(desc: EventDescription, message: string, level: Reporter::Level) &optional; The actual declaration uses the any type because of deficiencies of the Zeek type system.

config: table [string] of string &default = {  } &optional

A key/value table that will be passed to the reader. Interpretation of the values is left to the reader, but usually they will be used for configuration purposes.

An event input stream type used to send input data to a Zeek event.

Input::Mode
Type

enum

Input::MANUAL

Do not automatically reread the file after it has been read.

Input::REREAD

Reread the entire file each time a change is found.

Input::STREAM

Read data from end of file each time new data is appended.

Type that defines the input stream read mode.

Input::TableDescription
Type

record

source: string

String that allows the reader to find the source of the data. For READER_ASCII, this is the filename.

reader: Input::Reader &default = Input::default_reader &optional

Reader to use for this stream.

mode: Input::Mode &default = Input::default_mode &optional

Read mode to use for this stream.

name: string

Name of the input stream. This is used by some functions to manipulate the stream.

destination: any

Table which will receive the data read by the input framework.

idx: any

Record that defines the values used as the index of the table.

val: any &optional

Record that defines the values used as the elements of the table. If this is undefined, then destination must be a set.

want_record: bool &default = T &optional

Defines if the value of the table is a record (default), or a single value. When this is set to false, then val can only contain one element.

ev: any &optional

The event that is raised each time a value is added to, changed in, or removed from the table. The event will receive an Input::TableDescription as the first argument, an Input::Event enum as the second argument, the idx record as the third argument and the value (record) as the fourth argument.

pred: function (typ: Input::Event, left: any, right: any)bool &optional

Predicate function that can decide if an insertion, update or removal should really be executed. Parameters have same meaning as for the event. If true is returned, the update is performed. If false is returned, it is skipped.

error_ev: any &optional

Error event that is raised when an information, warning or error is raised by the input stream. If the level is error, the stream will automatically be closed. The event receives the Input::TableDescription as the first argument, the message as the second argument and the Reporter::Level as the third argument.

The event is raised like if it had been declared as follows: error_ev: function(desc: TableDescription, message: string, level: Reporter::Level) &optional; The actual declaration uses the any type because of deficiencies of the Zeek type system.

config: table [string] of string &default = {  } &optional

A key/value table that will be passed to the reader. Interpretation of the values is left to the reader, but usually they will be used for configuration purposes.

A table input stream type used to send data to a Zeek table.

Input::Reader
Type

enum

Input::READER_ASCII
Input::READER_BENCHMARK
Input::READER_BINARY
Input::READER_CONFIG
Input::READER_RAW
Input::READER_SQLITE

Events

Input::end_of_data
Type

event (name: string, source: string)

Event that is called when the end of a data source has been reached, including after an update.

Parameters
  • name – Name of the input stream.

  • source – String that identifies the data source (such as the filename).

Functions

Input::add_analysis
Type

function (description: Input::AnalysisDescription) : bool

Create a new file analysis input stream from a given source. Data read from the source is automatically forwarded to the file analysis framework.

Parameters

description – A record describing the source.

Returns

true on success.

Input::add_event
Type

function (description: Input::EventDescription) : bool

Create a new event input stream from a given source.

Parameters

descriptionEventDescription record describing the source.

Returns

true on success.

Input::add_table
Type

function (description: Input::TableDescription) : bool

Create a new table input stream from a given source.

Parameters

descriptionTableDescription record describing the source.

Returns

true on success.

Input::force_update
Type

function (id: string) : bool

Forces the current input to be checked for changes.

Parameters

id – string value identifying the stream.

Returns

true on success and false if the named stream was not found.

Input::remove
Type

function (id: string) : bool

Remove an input stream.

Parameters

id – string value identifying the stream to be removed.

Returns

true on success and false if the named stream was not found.