base/frameworks/intel/main.zeek
- Intel
The intelligence framework provides a way to store and query intelligence data (e.g. IP addresses, URLs and hashes). The intelligence items can be associated with metadata to allow informed decisions about matching and handling.
- Namespace:
Intel
- Imports:
Summary
Redefinable Options
The expiration timeout for intelligence items. |
Types
Record used for the logging framework representing a positive hit within the intelligence framework. |
|
Represents a piece of intelligence. |
|
Data about an |
|
Information about a piece of “seen” data. |
|
Enum type to represent various types of intelligence data. |
|
Set of intelligence data types. |
|
Enum to represent where data came from when it was discovered. |
Redefinitions
Events
Event to represent a match in the intelligence data from data that was seen. |
Hooks
This hook can be used to influence the logging of intelligence hits (e.g. |
|
This hook can be used to filter intelligence items that are about to be inserted into the internal data store. |
|
This hook can be used to handle expiration of intelligence items. |
|
Hook to modify and intercept |
Functions
Function to insert intelligence data. |
|
Function to remove intelligence data. |
|
Function to declare discovery of a piece of data in order to check it against known intelligence for matches. |
Detailed Interface
Redefinable Options
- Intel::item_expiration
- Type:
- Attributes:
- Default:
-1.0 min
- Redefinition:
from policy/frameworks/intel/do_expire.zeek
=
:10.0 mins
The expiration timeout for intelligence items. Once an item expires, the
Intel::item_expired
hook is called. Reinsertion of an item resets the timeout. A negative value disables expiration of intelligence items.
Types
- Intel::Info
- Type:
-
- ts:
time
&log
Timestamp when the data was discovered.
- uid:
string
&log
&optional
If a connection was associated with this intelligence hit, this is the uid for the connection
- id:
conn_id
&log
&optional
If a connection was associated with this intelligence hit, this is the conn_id for the connection.
- seen:
Intel::Seen
&log
Where the data was seen.
- matched:
Intel::TypeSet
&log
Which indicator types matched.
- sources:
set
[string
]&log
&default
={ }
&optional
Sources which supplied data that resulted in this match.
- fuid:
string
&log
&optional
(present if base/frameworks/intel/files.zeek is loaded)
If a file was associated with this intelligence hit, this is the uid for the file.
- file_mime_type:
string
&log
&optional
(present if base/frameworks/intel/files.zeek is loaded)
A mime type if the intelligence hit is related to a file. If the $f field is provided this will be automatically filled out.
- file_desc:
string
&log
&optional
(present if base/frameworks/intel/files.zeek is loaded)
Frequently files can be “described” to give a bit more context. If the $f field is provided this field will be automatically filled out.
- cif:
Intel::CIF
&log
&optional
(present if policy/integration/collective-intel/main.zeek is loaded)
- ts:
Record used for the logging framework representing a positive hit within the intelligence framework.
- Intel::Item
- Type:
-
- indicator:
string
The intelligence indicator.
- indicator_type:
Intel::Type
The type of data that the indicator field represents.
- meta:
Intel::MetaData
Metadata for the item. Typically represents more deeply descriptive data for a piece of intelligence.
- indicator:
Represents a piece of intelligence.
- Intel::MetaData
- Type:
-
- source:
string
An arbitrary string value representing the data source. This value is used as unique key to identify a metadata record in the scope of a single intelligence item.
- desc:
string
&optional
A freeform description for the data.
- url:
string
&optional
A URL for more information about the data.
- do_notice:
bool
&default
=F
&optional
(present if policy/frameworks/intel/do_notice.zeek is loaded)
A boolean value to allow the data itself to represent if the indicator that this metadata is attached to is notice worthy.
- if_in:
Intel::Where
&optional
(present if policy/frameworks/intel/do_notice.zeek is loaded)
Restrictions on when notices are created to only create them if the do_notice field is T and the notice was seen in the indicated location.
- whitelist:
bool
&default
=F
&optional
(present if policy/frameworks/intel/whitelist.zeek is loaded)
A boolean value to indicate whether the item is whitelisted.
- remove:
bool
&default
=F
&optional
(present if policy/frameworks/intel/removal.zeek is loaded)
A boolean value to indicate whether the item should be removed.
- cif_tags:
string
&optional
(present if policy/integration/collective-intel/main.zeek is loaded)
Maps to the ‘tags’ fields in CIF
- cif_confidence:
double
&optional
(present if policy/integration/collective-intel/main.zeek is loaded)
Maps to the ‘confidence’ field in CIF
- cif_source:
string
&optional
(present if policy/integration/collective-intel/main.zeek is loaded)
Maps to the ‘source’ field in CIF
- cif_description:
string
&optional
(present if policy/integration/collective-intel/main.zeek is loaded)
Maps to the ‘description’ field in CIF
- cif_firstseen:
string
&optional
(present if policy/integration/collective-intel/main.zeek is loaded)
Maps to the ‘firstseen’ field in CIF
- cif_lastseen:
string
&optional
(present if policy/integration/collective-intel/main.zeek is loaded)
Maps to the ‘lastseen’ field in CIF
- source:
Data about an
Intel::Item
.
- Intel::Seen
- Type:
-
- indicator:
string
&log
&optional
The string if the data is about a string.
- indicator_type:
Intel::Type
&log
&optional
The type of data that the indicator represents.
- host:
addr
&optional
If the indicator type was
Intel::ADDR
, then this field will be present.- where:
Intel::Where
&log
Where the data was discovered.
- node:
string
&optional
&log
The name of the node where the match was discovered.
- conn:
connection
&optional
If the data was discovered within a connection, the connection record should go here to give context to the data.
- uid:
string
&optional
If the data was discovered within a connection, the connection uid should go here to give context to the data. If the conn field is provided, this will be automatically filled out.
- f:
fa_file
&optional
(present if base/frameworks/intel/files.zeek is loaded)
If the data was discovered within a file, the file record should go here to provide context to the data.
- fuid:
string
&optional
(present if base/frameworks/intel/files.zeek is loaded)
If the data was discovered within a file, the file uid should go here to provide context to the data. If the file record f is provided, this will be automatically filled out.
- indicator:
Information about a piece of “seen” data.
- Intel::Type
- Type:
-
- Intel::ADDR
An IP address.
- Intel::SUBNET
A subnet in CIDR notation.
- Intel::URL
A complete URL without the prefix
"http://"
.
- Intel::SOFTWARE
Software name.
- Intel::EMAIL
Email address.
- Intel::DOMAIN
DNS domain name.
- Intel::USER_NAME
A user name.
- Intel::CERT_HASH
Certificate SHA-1 hash.
- Intel::PUBKEY_HASH
Public key MD5 hash, formatted as hexadecimal digits delimited by colons. (SSH server host keys are a good example.)
- Intel::FILE_HASH
(present if base/frameworks/intel/files.zeek is loaded)
File hash which is non-hash type specific. It’s up to the user to query for any relevant hash types.
- Intel::FILE_NAME
(present if base/frameworks/intel/files.zeek is loaded)
File name. Typically with protocols with definite indications of a file name.
Enum type to represent various types of intelligence data.
- Intel::TypeSet
- Type:
Set of intelligence data types.
- Intel::Where
- Type:
-
- Intel::IN_ANYWHERE
A catchall value to represent data of unknown provenance.
- Conn::IN_ORIG
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- Conn::IN_RESP
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- Files::IN_HASH
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- Files::IN_NAME
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- DNS::IN_REQUEST
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- DNS::IN_RESPONSE
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- HTTP::IN_HOST_HEADER
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- HTTP::IN_REFERRER_HEADER
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- HTTP::IN_USER_AGENT_HEADER
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- HTTP::IN_X_FORWARDED_FOR_HEADER
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- HTTP::IN_URL
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMTP::IN_MAIL_FROM
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMTP::IN_RCPT_TO
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMTP::IN_FROM
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMTP::IN_TO
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMTP::IN_CC
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMTP::IN_RECEIVED_HEADER
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMTP::IN_REPLY_TO
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMTP::IN_X_ORIGINATING_IP_HEADER
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMTP::IN_MESSAGE
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SSH::IN_SERVER_HOST_KEY
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SSL::IN_SERVER_NAME
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMTP::IN_HEADER
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- X509::IN_CERT
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SMB::IN_FILE_NAME
(present if policy/frameworks/intel/seen/where-locations.zeek is loaded)
- SSH::SUCCESSFUL_LOGIN
(present if policy/protocols/ssh/detect-bruteforcing.zeek is loaded)
An indicator of the login for the intel framework.
Enum to represent where data came from when it was discovered. The convention is to prefix the name with
IN_
.
Events
- Intel::log_intel
- Type:
event
(rec:Intel::Info
)
- Intel::match
- Type:
event
(s:Intel::Seen
, items:set
[Intel::Item
])
Event to represent a match in the intelligence data from data that was seen. On clusters there is no assurance as to when this event will be generated so do not assume that arbitrary global state beyond the given data will be available.
This is the primary mechanism where a user may take actions based on data provided by the intelligence framework.
Hooks
- Intel::extend_match
- Type:
hook
(info:Intel::Info
, s:Intel::Seen
, items:set
[Intel::Item
]) :bool
This hook can be used to influence the logging of intelligence hits (e.g. by adding data to the Info record). The default information is added with a priority of 5.
- Parameters:
info – The Info record that will be logged.
s – Information about the data seen.
items – The intel items that match the seen data.
In case the hook execution is terminated using break, the match will not be logged.
- Intel::filter_item
- Type:
hook
(item:Intel::Item
) :bool
This hook can be used to filter intelligence items that are about to be inserted into the internal data store. In case the hook execution is terminated using break, the item will not be (re)added to the internal data store.
- Parameters:
item – The intel item that should be inserted.
- Intel::item_expired
- Type:
hook
(indicator:string
, indicator_type:Intel::Type
, metas:set
[Intel::MetaData
]) :bool
This hook can be used to handle expiration of intelligence items.
- Parameters:
indicator – The indicator of the expired item.
indicator_type – The indicator type of the expired item.
metas – The set of metadata describing the expired item.
If all hook handlers are executed, the expiration timeout will be reset. Otherwise, if one of the handlers terminates using break, the item will be removed.
- Intel::seen_policy
- Type:
hook
(s:Intel::Seen
, found:bool
) :bool
Hook to modify and intercept
Intel::seen
behavior.This hook is invoked after the Intel datastore was searched for a given
Intel::Seen
instance. If a matching entry was found, the found argument is set toT
, elseF
.Breaking from this hook suppresses
Intel::match
event generation and any subsequent logging.Note that this hook only runs on the Zeek node where
Intel::seen
is invoked. In a cluster configuration that is usually on the worker nodes. This is in contrast toIntel::match
that usually runs centrally on the the manager node instead.- Parameters:
s – The
Intel::Seen
instance passed to theIntel::seen
function.found –
T
if Intel datastore contained s, elseF
.
Functions
- Intel::insert
- Type:
function
(item:Intel::Item
) :void
Function to insert intelligence data. If the indicator is already present, the associated metadata will be added to the indicator. If the indicator already contains a metadata record from the same source, the existing metadata record will be updated.
- Intel::remove
-
Function to remove intelligence data. If purge_indicator is set, the given metadata is ignored and the indicator is removed completely.
- Intel::seen
- Type:
function
(s:Intel::Seen
) :void
Function to declare discovery of a piece of data in order to check it against known intelligence for matches.