base/frameworks/notice/main.zeek

GLOBAL
Notice

This is the notice framework which enables Zeek to “notice” things which are odd or potentially bad. Decisions of the meaning of various notices need to be done per site because Zeek does not ship with assumptions about what is bad activity for sites. More extensive documentation about using the notice framework can be found in Notice Framework.

Namespaces

GLOBAL, Notice

Imports

base/frameworks/cluster

Summary

Runtime Options

Notice::alarmed_types: set &redef

Alarmed notice types.

Notice::default_suppression_interval: interval &redef

The notice framework is able to do automatic notice suppression by utilizing the identifier field in Notice::Info records.

Notice::emailed_types: set &redef

Emailed notice types.

Notice::ignored_types: set &redef

Ignored notice types.

Notice::mail_from: string &redef

Address that emails will be from.

Notice::mail_subject_prefix: string &redef

Text string prefixed to the subject of all emails sent out.

Notice::not_suppressed_types: set &redef

Types that should be suppressed for the default suppression interval.

Notice::reply_to: string &redef

Reply-to address used in outbound email.

Notice::sendmail: string &redef

Local system sendmail program.

Redefinable Options

Notice::mail_dest: string &redef

The default email address to send notices with the Notice::ACTION_EMAIL action or to send bulk alarm logs on rotation with Notice::ACTION_ALARM.

Notice::max_email_delay: interval &redef

The maximum amount of time a plugin can delay email from being sent.

Notice::type_suppression_intervals: table &redef

This table can be used as a shorthand way to modify suppression intervals for entire notice types.

Types

Notice::Action: enum

These are values representing actions that can be taken with notices.

Notice::ActionSet: set

Type that represents a set of actions.

Notice::FileInfo: record

Contains a portion of fa_file that’s also contained in Notice::Info.

Notice::Info: record

The record type that is used for representing and logging notices.

Notice::Type: enum

Scripts creating new notices need to redef this enum to add their own specific notice types which would then get used when they call the NOTICE function.

Redefinitions

Log::ID: enum

Events

Notice::begin_suppression: event

This event is generated when a notice begins to be suppressed.

Notice::log_notice: event

This event can be handled to access the Notice::Info record as it is sent on to the logging framework.

Notice::manager_begin_suppression: event

This is an internal event that is used to broadcast the begin_suppression event over a cluster.

Notice::suppressed: event

This event is generated on each occurrence of an event being suppressed.

Hooks

Notice::log_policy: Log::PolicyHook

Default logging policy hooks for the streams.

Notice::log_policy_alarm: Log::PolicyHook

Notice::notice: hook

This is the event that is called as the entry point to the notice framework by the global NOTICE function.

Notice::policy: hook

The hook to modify notice handling.

Functions

NOTICE: function

Notice::apply_policy: function

This is an internal function to populate policy records.

Notice::create_file_info: function

Creates a record containing a subset of a full fa_file record.

Notice::email_headers: function

Constructs mail headers to which an email body can be appended for sending with sendmail.

Notice::email_notice_to: function

Call this function to send a notice in an email.

Notice::is_being_suppressed: function

A function to determine if an event is supposed to be suppressed.

Notice::log_mailing_postprocessor: function

A log postprocessing function that implements emailing the contents of a log upon rotation to any configured Notice::mail_dest.

Notice::populate_file_info: function

Populates file-related fields in a notice info record.

Notice::populate_file_info2: function

Populates file-related fields in a notice info record.

Detailed Interface

Runtime Options

Notice::alarmed_types
Type

set [Notice::Type]

Attributes

&redef

Default

{}

Alarmed notice types.

Notice::default_suppression_interval
Type

interval

Attributes

&redef

Default

1.0 hr

The notice framework is able to do automatic notice suppression by utilizing the identifier field in Notice::Info records. Set this to “0secs” to completely disable automated notice suppression.

Notice::emailed_types
Type

set [Notice::Type]

Attributes

&redef

Default

{}

Emailed notice types.

Notice::ignored_types
Type

set [Notice::Type]

Attributes

&redef

Default

{}

Ignored notice types.

Notice::mail_from
Type

string

Attributes

&redef

Default

"Zeek <zeek@localhost>"

Address that emails will be from.

Note that this is overridden by the ZeekControl MailFrom option.

Notice::mail_subject_prefix
Type

string

Attributes

&redef

Default

"[Zeek]"

Text string prefixed to the subject of all emails sent out.

Note that this is overridden by the ZeekControl MailSubjectPrefix option.

Notice::not_suppressed_types
Type

set [Notice::Type]

Attributes

&redef

Default

{}

Types that should be suppressed for the default suppression interval.

Notice::reply_to
Type

string

Attributes

&redef

Default

""

Reply-to address used in outbound email.

Notice::sendmail
Type

string

Attributes

&redef

Default

"/usr/sbin/sendmail"

Local system sendmail program.

Note that this is overridden by the ZeekControl SendMail option.

Redefinable Options

Notice::mail_dest
Type

string

Attributes

&redef

Default

""

The default email address to send notices with the Notice::ACTION_EMAIL action or to send bulk alarm logs on rotation with Notice::ACTION_ALARM.

Note that this is overridden by the ZeekControl MailTo option or by the email_dest field in the Notice::Info record.

Notice::max_email_delay
Type

interval

Attributes

&redef

Default

15.0 secs

The maximum amount of time a plugin can delay email from being sent.

Notice::type_suppression_intervals
Type

table [Notice::Type] of interval

Attributes

&redef

Default

{}

This table can be used as a shorthand way to modify suppression intervals for entire notice types.

Types

Notice::Action
Type

enum

Notice::ACTION_NONE

Indicates that there is no action to be taken.

Notice::ACTION_LOG

Indicates that the notice should be sent to the notice logging stream.

Notice::ACTION_EMAIL

Indicates that the notice should be sent to the email address(es) configured in the Notice::mail_dest variable.

Notice::ACTION_ALARM

Indicates that the notice should be alarmed. A readable ASCII version is saved in notice_alarm log, and emailed in bulk to the address(es) configured in Notice::mail_dest.

Notice::ACTION_DROP

Indicates that the notice should result in a drop action. The exact action taken depends on loaded policy scripts; see e.g. NetControl::acld_rule_policy.

Notice::ACTION_EMAIL_ADMIN

(present if base/frameworks/notice/actions/email_admin.zeek is loaded)

Indicate that the generated email should be addressed to the appropriate email addresses as found by the Site::get_emails function based on the relevant address or addresses indicated in the notice.

Notice::ACTION_PAGE

(present if base/frameworks/notice/actions/page.zeek is loaded)

Indicates that the notice should be sent to the pager email address configured in the Notice::mail_page_dest variable.

Notice::ACTION_ADD_GEODATA

(present if base/frameworks/notice/actions/add-geodata.zeek is loaded)

Indicates that the notice should have geodata added for the “remote” host. Site::local_nets must be defined in order for this to work.

These are values representing actions that can be taken with notices.

Notice::ActionSet
Type

set [Notice::Action]

Type that represents a set of actions.

Notice::FileInfo
Type

record

fuid: string

File UID.

desc: string

File description from e.g. Files::describe.

mime: string &optional

Strongest mime type match for file.

cid: conn_id &optional

Connection tuple over which file is sent.

cuid: string &optional

Connection UID over which file is sent.

Contains a portion of fa_file that’s also contained in Notice::Info.

Notice::Info
Type

record

ts: time &log &optional

An absolute time indicating when the notice occurred, defaults to the current network time.

uid: string &log &optional

A connection UID which uniquely identifies the endpoints concerned with the notice.

id: conn_id &log &optional

A connection 4-tuple identifying the endpoints concerned with the notice.

conn: connection &optional

A shorthand way of giving the uid and id to a notice. The reference to the actual connection will be deleted after applying the notice policy.

iconn: icmp_conn &optional

A shorthand way of giving the uid and id to a notice. The reference to the actual connection will be deleted after applying the notice policy.

f: fa_file &optional

A file record if the notice is related to a file. The reference to the actual fa_file record will be deleted after applying the notice policy.

fuid: string &log &optional

A file unique ID if this notice is related to a file. If the f field is provided, this will be automatically filled out.

file_mime_type: string &log &optional

A mime type if the notice is related to a file. If the f field is provided, this will be automatically filled out.

file_desc: string &log &optional

Frequently files can be “described” to give a bit more context. This field will typically be automatically filled out from an fa_file record. For example, if a notice was related to a file over HTTP, the URL of the request would be shown.

proto: transport_proto &log &optional

The transport protocol. Filled automatically when either conn, iconn or p is specified.

note: Notice::Type &log

The Notice::Type of the notice.

msg: string &log &optional

The human readable message for the notice.

sub: string &log &optional

The human readable sub-message.

src: addr &log &optional

Source address, if we don’t have a conn_id.

dst: addr &log &optional

Destination address.

p: port &log &optional

Associated port, if we don’t have a conn_id.

n: count &log &optional

Associated count, or perhaps a status code.

peer_name: string &optional

Name of remote peer that raised this notice.

peer_descr: string &log &optional

Textual description for the peer that raised this notice, including name, host address and port.

actions: Notice::ActionSet &log &default = {  } &optional

The actions which have been applied to this notice.

email_dest: set [string] &log &default = {  } &optional

The email address(es) where to send this notice

email_body_sections: vector of string &optional

By adding chunks of text into this element, other scripts can expand on notices that are being emailed. The normal way to add text is to extend the vector by handling the Notice::notice event and modifying the notice in place.

email_delay_tokens: set [string] &optional

Adding a string “token” to this set will cause the notice framework’s built-in emailing functionality to delay sending the email until either the token has been removed or the email has been delayed for Notice::max_email_delay.

identifier: string &optional

This field is to be provided when a notice is generated for the purpose of deduplicating notices. The identifier string should be unique for a single instance of the notice. This field should be filled out in almost all cases when generating notices to define when a notice is conceptually a duplicate of a previous notice.

For example, an SSL certificate that is going to expire soon should always have the same identifier no matter the client IP address that connected and resulted in the certificate being exposed. In this case, the resp_h, resp_p, and hash of the certificate would be used to create this value. The hash of the cert is included because servers can return multiple certificates on the same port.

Another example might be a host downloading a file which triggered a notice because the MD5 sum of the file it downloaded was known by some set of intelligence. In that case, the orig_h (client) and MD5 sum would be used in this field to dedup because if the same file is downloaded over and over again you really only want to know about it a single time. This makes it possible to send those notices to email without worrying so much about sending thousands of emails.

suppress_for: interval &log &default = Notice::default_suppression_interval &optional

This field indicates the length of time that this unique notice should be suppressed.

remote_location: geo_location &log &optional

(present if base/frameworks/notice/actions/add-geodata.zeek is loaded)

If GeoIP support is built in, notices can have geographic information attached to them.

dropped: bool &log &default = F &optional

(present if policy/frameworks/notice/actions/drop.zeek is loaded)

Indicate if the $src IP address was dropped and denied network access.

community_id: string &optional &log

(present if policy/frameworks/notice/community-id.zeek is loaded)

The record type that is used for representing and logging notices.

Notice::Type
Type

enum

Notice::Tally

Notice reporting a count of how often a notice occurred.

Weird::Activity

(present if base/frameworks/notice/weird.zeek is loaded)

Generic unusual but notice-worthy weird activity.

Signatures::Sensitive_Signature

(present if base/frameworks/signatures/main.zeek is loaded)

Generic notice type for notice-worthy signature matches.

Signatures::Multiple_Signatures

(present if base/frameworks/signatures/main.zeek is loaded)

Host has triggered many signatures on the same host. The number of signatures is defined by the Signatures::vert_scan_thresholds variable.

Signatures::Multiple_Sig_Responders

(present if base/frameworks/signatures/main.zeek is loaded)

Host has triggered the same signature on multiple hosts as defined by the Signatures::horiz_scan_thresholds variable.

Signatures::Count_Signature

(present if base/frameworks/signatures/main.zeek is loaded)

The same signature has triggered multiple times for a host. The number of times the signature has been triggered is defined by the Signatures::count_thresholds variable. To generate this notice, the Signatures::SIG_COUNT_PER_RESP action must be set for the signature.

Signatures::Signature_Summary

(present if base/frameworks/signatures/main.zeek is loaded)

Summarize the number of times a host triggered a signature. The interval between summaries is defined by the Signatures::summary_interval variable.

PacketFilter::Compile_Failure

(present if base/frameworks/packet-filter/main.zeek is loaded)

This notice is generated if a packet filter cannot be compiled.

PacketFilter::Install_Failure

(present if base/frameworks/packet-filter/main.zeek is loaded)

Generated if a packet filter fails to install.

PacketFilter::Too_Long_To_Compile_Filter

(present if base/frameworks/packet-filter/main.zeek is loaded)

Generated when a notice takes too long to compile.

PacketFilter::Dropped_Packets

(present if base/frameworks/packet-filter/netstats.zeek is loaded)

Indicates packets were dropped by the packet filter.

ProtocolDetector::Protocol_Found

(present if policy/frameworks/dpd/detect-protocols.zeek is loaded)

ProtocolDetector::Server_Found

(present if policy/frameworks/dpd/detect-protocols.zeek is loaded)

Intel::Notice

(present if policy/frameworks/intel/do_notice.zeek is loaded)

This notice is generated when an intelligence indicator is denoted to be notice-worthy.

TeamCymruMalwareHashRegistry::Match

(present if policy/frameworks/files/detect-MHR.zeek is loaded)

The hash value of a file transferred over HTTP matched in the malware hash registry.

PacketFilter::No_More_Conn_Shunts_Available

(present if policy/frameworks/packet-filter/shunt.zeek is loaded)

Indicative that PacketFilter::max_bpf_shunts connections are already being shunted with BPF filters and no more are allowed.

PacketFilter::Cannot_BPF_Shunt_Conn

(present if policy/frameworks/packet-filter/shunt.zeek is loaded)

Limitations in BPF make shunting some connections with BPF impossible. This notice encompasses those various cases.

Software::Software_Version_Change

(present if policy/frameworks/software/version-changes.zeek is loaded)

For certain software, a version changing may matter. In that case, this notice will be generated. Software that matters if the version changes can be configured with the Software::interesting_version_changes variable.

Software::Vulnerable_Version

(present if policy/frameworks/software/vulnerable.zeek is loaded)

Indicates that a vulnerable version of software was detected.

CaptureLoss::Too_Much_Loss

(present if policy/misc/capture-loss.zeek is loaded)

Report if the detected capture loss exceeds the percentage threshold defined in CaptureLoss::too_much_loss.

CaptureLoss::Too_Little_Traffic

(present if policy/misc/capture-loss.zeek is loaded)

Report if the traffic seen by a peer within a given watch interval is less than CaptureLoss::minimum_acks.

Traceroute::Detected

(present if policy/misc/detect-traceroute/main.zeek is loaded)

Indicates that a host was seen running traceroutes. For more detail about specific traceroutes that we run, refer to the traceroute.log.

Scan::Address_Scan

(present if policy/misc/scan.zeek is loaded)

Address scans detect that a host appears to be scanning some number of destinations on a single port. This notice is generated when more than Scan::addr_scan_threshold unique hosts are seen over the previous Scan::addr_scan_interval time range.

Scan::Port_Scan

(present if policy/misc/scan.zeek is loaded)

Port scans detect that an attacking host appears to be scanning a single victim host on several ports. This notice is generated when an attacking host attempts to connect to Scan::port_scan_threshold unique ports on a single host over the previous Scan::port_scan_interval time range.

Conn::Retransmission_Inconsistency

(present if policy/protocols/conn/weirds.zeek is loaded)

Possible evasion; usually just chud.

Conn::Content_Gap

(present if policy/protocols/conn/weirds.zeek is loaded)

Data has sequence hole; perhaps due to filtering.

DNS::External_Name

(present if policy/protocols/dns/detect-external-names.zeek is loaded)

Raised when a non-local name is found to be pointing at a local host. The Site::local_zones variable must be set appropriately for this detection.

FTP::Bruteforcing

(present if policy/protocols/ftp/detect-bruteforcing.zeek is loaded)

Indicates a host bruteforcing FTP logins by watching for too many rejected usernames or failed passwords.

FTP::Site_Exec_Success

(present if policy/protocols/ftp/detect.zeek is loaded)

Indicates that a successful response to a “SITE EXEC” command/arg pair was seen.

HTTP::SQL_Injection_Attacker

(present if policy/protocols/http/detect-sqli.zeek is loaded)

Indicates that a host performing SQL injection attacks was detected.

HTTP::SQL_Injection_Victim

(present if policy/protocols/http/detect-sqli.zeek is loaded)

Indicates that a host was seen to have SQL injection attacks against it. This is tracked by IP address as opposed to hostname.

SMTP::Blocklist_Error_Message

(present if policy/protocols/smtp/blocklists.zeek is loaded)

An SMTP server sent a reply mentioning an SMTP block list.

SMTP::Blocklist_Blocked_Host

(present if policy/protocols/smtp/blocklists.zeek is loaded)

The originator’s address is seen in the block list error message. This is useful to detect local hosts sending SPAM with a high positive rate.

SMTP::Suspicious_Origination

(present if policy/protocols/smtp/detect-suspicious-orig.zeek is loaded)

SSH::Password_Guessing

(present if policy/protocols/ssh/detect-bruteforcing.zeek is loaded)

Indicates that a host has been identified as crossing the SSH::password_guesses_limit threshold with failed logins.

SSH::Login_By_Password_Guesser

(present if policy/protocols/ssh/detect-bruteforcing.zeek is loaded)

Indicates that a host previously identified as a “password guesser” has now had a successful login attempt. This is not currently implemented.

SSH::Watched_Country_Login

(present if policy/protocols/ssh/geo-data.zeek is loaded)

If an SSH login is seen to or from a “watched” country based on the SSH::watched_countries variable then this notice will be generated.

SSH::Interesting_Hostname_Login

(present if policy/protocols/ssh/interesting-hostnames.zeek is loaded)

Generated if a login originates or responds with a host where the reverse hostname lookup resolves to a name matched by the SSH::interesting_hostnames regular expression.

SSL::Certificate_Expired

(present if policy/protocols/ssl/expiring-certs.zeek is loaded)

Indicates that a certificate’s NotValidAfter date has lapsed and the certificate is now invalid.

SSL::Certificate_Expires_Soon

(present if policy/protocols/ssl/expiring-certs.zeek is loaded)

Indicates that a certificate is going to expire within SSL::notify_when_cert_expiring_in.

SSL::Certificate_Not_Valid_Yet

(present if policy/protocols/ssl/expiring-certs.zeek is loaded)

Indicates that a certificate’s NotValidBefore date is future dated.

Heartbleed::SSL_Heartbeat_Attack

(present if policy/protocols/ssl/heartbleed.zeek is loaded)

Indicates that a host performed a heartbleed attack or scan.

Heartbleed::SSL_Heartbeat_Attack_Success

(present if policy/protocols/ssl/heartbleed.zeek is loaded)

Indicates that a host performing a heartbleed attack was probably successful.

Heartbleed::SSL_Heartbeat_Odd_Length

(present if policy/protocols/ssl/heartbleed.zeek is loaded)

Indicates we saw heartbeat requests with odd length. Probably an attack or scan.

Heartbleed::SSL_Heartbeat_Many_Requests

(present if policy/protocols/ssl/heartbleed.zeek is loaded)

Indicates we saw many heartbeat requests without a reply. Might be an attack.

SSL::Invalid_Server_Cert

(present if policy/protocols/ssl/validate-certs.zeek is loaded)

This notice indicates that the result of validating the certificate along with its full certificate chain was invalid.

SSL::Invalid_Ocsp_Response

(present if policy/protocols/ssl/validate-ocsp.zeek is loaded)

This indicates that the OCSP response was not deemed to be valid.

SSL::Weak_Key

(present if policy/protocols/ssl/weak-keys.zeek is loaded)

Indicates that a server is using a potentially unsafe key.

SSL::Old_Version

(present if policy/protocols/ssl/weak-keys.zeek is loaded)

Indicates that a server is using a potentially unsafe version

SSL::Weak_Cipher

(present if policy/protocols/ssl/weak-keys.zeek is loaded)

Indicates that a server is using a potentially unsafe cipher

ZeekygenExample::Zeekygen_One

(present if zeekygen/example.zeek is loaded)

Any number of this type of comment will document “Zeekygen_One”.

ZeekygenExample::Zeekygen_Two

(present if zeekygen/example.zeek is loaded)

Any number of this type of comment will document “ZEEKYGEN_TWO”.

ZeekygenExample::Zeekygen_Three

(present if zeekygen/example.zeek is loaded)

ZeekygenExample::Zeekygen_Four

(present if zeekygen/example.zeek is loaded)

Omitting comments is fine, and so is mixing ## and ##<, but it’s probably best to use only one style consistently.

Scripts creating new notices need to redef this enum to add their own specific notice types which would then get used when they call the NOTICE function. The convention is to give a general category along with the specific notice separating words with underscores and using leading capitals on each word except for abbreviations which are kept in all capitals. For example, SSH::Password_Guessing is for hosts that have crossed a threshold of failed SSH logins.

Events

Notice::begin_suppression
Type

event (ts: time, suppress_for: interval, note: Notice::Type, identifier: string)

This event is generated when a notice begins to be suppressed.

Parameters
  • ts – time indicating then when the notice to be suppressed occurred.

  • suppress_for – length of time that this notice should be suppressed.

  • note – The Notice::Type of the notice.

  • identifier – The identifier string of the notice that should be suppressed.

Notice::log_notice
Type

event (rec: Notice::Info)

This event can be handled to access the Notice::Info record as it is sent on to the logging framework.

Parameters

rec – The record containing notice data before it is logged.

Notice::manager_begin_suppression
Type

event (ts: time, suppress_for: interval, note: Notice::Type, identifier: string)

This is an internal event that is used to broadcast the begin_suppression event over a cluster.

Parameters
  • ts – time indicating then when the notice to be suppressed occurred.

  • suppress_for – length of time that this notice should be suppressed.

  • note – The Notice::Type of the notice.

  • identifier – The identifier string of the notice that should be suppressed.

Notice::suppressed
Type

event (n: Notice::Info)

This event is generated on each occurrence of an event being suppressed.

Parameters

n – The record containing notice data regarding the notice type being suppressed.

Hooks

Notice::log_policy
Type

Log::PolicyHook

Default logging policy hooks for the streams.

Notice::log_policy_alarm
Type

Log::PolicyHook

Notice::notice
Type

hook (n: Notice::Info) : bool

This is the event that is called as the entry point to the notice framework by the global NOTICE function. By the time this event is generated, default values have already been filled out in the Notice::Info record and the notice policy has also been applied.

Parameters

n – The record containing notice data.

Notice::policy
Type

hook (n: Notice::Info) : bool

The hook to modify notice handling.

Functions

NOTICE
Type

function (n: Notice::Info) : void

Notice::apply_policy
Type

function (n: Notice::Info) : void

This is an internal function to populate policy records.

Notice::create_file_info
Type

function (f: fa_file) : Notice::FileInfo

Creates a record containing a subset of a full fa_file record.

Parameters

f – record containing metadata about a file.

Returns

record containing a subset of fields copied from f.

Notice::email_headers
Type

function (subject_desc: string, dest: string) : string

Constructs mail headers to which an email body can be appended for sending with sendmail.

Parameters
  • subject_desc – a subject string to use for the mail.

  • dest – recipient string to use for the mail.

Returns

a string of mail headers to which an email body can be appended.

Notice::email_notice_to
Type

function (n: Notice::Info, dest: string, extend: bool) : void

Call this function to send a notice in an email. It is already used by default with the built in Notice::ACTION_EMAIL and Notice::ACTION_PAGE actions.

Parameters
  • n – The record of notice data to email.

  • dest – The intended recipient of the notice email.

  • extend – Whether to extend the email using the email_body_sections field of n.

Notice::is_being_suppressed
Type

function (n: Notice::Info) : bool

A function to determine if an event is supposed to be suppressed.

Parameters

n – The record containing the notice in question.

Notice::log_mailing_postprocessor
Type

function (info: Log::RotationInfo) : bool

A log postprocessing function that implements emailing the contents of a log upon rotation to any configured Notice::mail_dest. The rotated log is removed upon being sent.

Parameters

info – A record containing the rotated log file information.

Returns

True.

Notice::populate_file_info
Type

function (f: fa_file, n: Notice::Info) : void

Populates file-related fields in a notice info record.

Parameters
  • f – record containing metadata about a file.

  • n – a notice record that needs file-related fields populated.

Notice::populate_file_info2
Type

function (fi: Notice::FileInfo, n: Notice::Info) : void

Populates file-related fields in a notice info record.

Parameters
  • fi – record containing metadata about a file.

  • n – a notice record that needs file-related fields populated.