known-services.zeek

Known

This script logs and tracks active services. For this script, an active service is defined as an IP address and port of a server for which a TCP handshake (SYN+ACK) is observed, assumed to have been done in the past (started seeing packets mid-connection, but the server is actively sending data), or sent at least one UDP packet. If a protocol name is found/known for service, that will be logged, but services whose names can’t be determined are also still logged.

Namespace:: Known
Imports:: base/frameworks/cluster, base/utils/directions-and-hosts.zeek

Summary

Runtime Options

`Known::service_store_timeout`: `interval` `&redef`	The timeout interval to use for operations against `Known::service_store`.
`Known::service_tracking`: `Host` `&redef`	The hosts whose services should be tracked and logged.
`Known::service_udp_requires_response`: `bool` `&redef`	Require UDP server to respond before considering it an “active service”.

Redefinable Options

`Known::service_store_expiry`: `interval` `&redef`	The expiry interval of new entries in `Known::service_store`.
`Known::service_store_name`: `string` `&redef`	The Broker topic name to use for `Known::service_store`.
`Known::use_service_store`: `bool` `&redef`	Toggles between different implementations of this script.

State Variables

`Known::service_store`: `Cluster::StoreInfo`	Holds the set of all known services.
`Known::services`: `table` `&create_expire` = `1.0 day`	Tracks the set of daily-detected services for preventing the logging of duplicates, but can also be inspected by other scripts for different purposes.

Types

`Known::AddrPortServTriplet`: `record`
`Known::ServicesInfo`: `record`	The record type which contains the column fields of the known-services log.

Redefinitions

Log::ID: enum

The known-services logging stream identifier.

Known::SERVICES_LOG

connection: record

New Fields:

connection

known_services_done: bool &default = F &optional

Events

Known::log_known_services: event

Event that can be handled to access the Known::ServicesInfo record as it is sent on to the logging framework.

Hooks

Known::log_policy_services: Log::PolicyHook

A default logging policy hook for the stream.

Detailed Interface

Runtime Options

Known::service_store_timeout 

Type:: interval
Attributes:: &redef
Default:: 15.0 secs

The timeout interval to use for operations against Known::service_store.

Known::service_tracking 

Type:

Host

Attributes:

&redef

Default:

LOCAL_HOSTS

Redefinition:

from policy/tuning/track-all-assets.zeek

=:

ALL_HOSTS

The hosts whose services should be tracked and logged. See Host for possible choices.

Known::service_udp_requires_response 

Type:: bool
Attributes:: &redef
Default:: T

Require UDP server to respond before considering it an “active service”.

Redefinable Options

Known::service_store_expiry 

Type:: interval
Attributes:: &redef
Default:: 1.0 day

The expiry interval of new entries in Known::service_store. This also changes the interval at which services get logged.

Known::service_store_name 

Type:: string
Attributes:: &redef
Default:: "zeek/known/services"

The Broker topic name to use for Known::service_store.

Known::use_service_store 

Type:: bool
Attributes:: &redef
Default:: F

Toggles between different implementations of this script. When true, use a Broker data store, else use a regular Zeek set with keys uniformly distributed over proxy nodes in cluster operation.

State Variables

Known::service_store 

Type:

Cluster::StoreInfo

Default:

{
   name=<uninitialized>
   store=<uninitialized>
   master_node=""
   master=F
   backend=Broker::MEMORY
   options=[sqlite=[path="", synchronous=<uninitialized>, journal_mode=<uninitialized>, failure_mode=Broker::SQLITE_FAILURE_MODE_FAIL, integrity_check=F]]
   clone_resync_interval=10.0 secs
   clone_stale_interval=5.0 mins
   clone_mutation_buffer_interval=2.0 mins
}

Holds the set of all known services. Keys in the store are Known::AddrPortServTriplet and their associated value is always the boolean value of “true”.

Known::services 

Type:: table [addr, port] of set [string]
Attributes:: &create_expire = 1.0 day
Default:: {}

Tracks the set of daily-detected services for preventing the logging of duplicates, but can also be inspected by other scripts for different purposes.

In cluster operation, this table is uniformly distributed across proxy nodes.

This table is automatically populated and shouldn’t be directly modified.

Types

Known::AddrPortServTriplet 

Type:

record

host: addr

p: port

serv: string

Known::ServicesInfo 

Type:

record

ts: time &log: The time at which the service was detected.
host: addr &log: The host address on which the service is running.
port_num: port &log: The port number on which the service is running.
port_proto: transport_proto &log: The transport-layer protocol which the service uses.
service: set [string] &log: A set of protocols that match the service’s connection payloads.

The record type which contains the column fields of the known-services log.

Events

Known::log_known_services 

Type:: event (rec: Known::ServicesInfo)

Event that can be handled to access the Known::ServicesInfo record as it is sent on to the logging framework.

Hooks

Known::log_policy_services 

Type:: Log::PolicyHook

A default logging policy hook for the stream.