base/protocols/http/main.zeek

HTTP

Implements base functionality for HTTP analysis. The logging model is to log request/response pairs and all relevant metadata together in a single record.

Namespace

HTTP

Imports

base/frameworks/tunnels, base/protocols/conn/removal-hooks.zeek, base/utils/files.zeek, base/utils/numbers.zeek

Summary

Runtime Options

HTTP::default_capture_password: bool &redef

This setting changes if passwords used in Basic-Auth are captured or not.

HTTP::http_methods: set &redef

A list of HTTP methods.

HTTP::max_pending_requests: count &redef

Only allow that many pending requests on a single connection.

HTTP::proxy_headers: set &redef

A list of HTTP headers typically used to indicate proxied requests.

Types

HTTP::Info: record

The record type which contains the fields of the HTTP log.

HTTP::State: record

Structure to maintain state for an HTTP connection with multiple requests and responses.

HTTP::Tags: enum

Indicate a type of attack or compromise in the record to be logged.

Redefinitions

Log::ID: enum

connection: record

New Fields

connection

http: HTTP::Info &optional

http_state: HTTP::State &optional

likely_server_ports: set &redef

Events

HTTP::log_http: event

Event that can be handled to access the HTTP record as it is sent on to the logging framework.

Hooks

HTTP::finalize_http: Conn::RemovalHook

HTTP finalization hook.

HTTP::log_policy: Log::PolicyHook

Detailed Interface

Runtime Options

HTTP::default_capture_password
Type

bool

Attributes

&redef

Default

F

This setting changes if passwords used in Basic-Auth are captured or not.

HTTP::http_methods
Type

set [string]

Attributes

&redef

Default
{
   "POST",
   "PUT",
   "CONNECT",
   "BMOVE",
   "SEARCH",
   "TRACE",
   "LOCK",
   "PROPPATCH",
   "HEAD",
   "OPTIONS",
   "POLL",
   "REPORT",
   "SUBSCRIBE",
   "MOVE",
   "GET",
   "UNLOCK",
   "DELETE",
   "COPY",
   "MKCOL",
   "PROPFIND"
}

A list of HTTP methods. Other methods will generate a weird. Note that the HTTP analyzer will only accept methods consisting solely of letters [A-Za-z].

HTTP::max_pending_requests
Type

count

Attributes

&redef

Default

100

Only allow that many pending requests on a single connection. If this number is exceeded, all pending requests are flushed out and request/response tracking reset to prevent unbounded state growth.

HTTP::proxy_headers
Type

set [string]

Attributes

&redef

Default
{
   "CLIENT-IP",
   "X-FORWARDED-FROM",
   "VIA",
   "XROXY-CONNECTION",
   "PROXY-CONNECTION",
   "X-FORWARDED-FOR",
   "FORWARDED"
}

A list of HTTP headers typically used to indicate proxied requests.

Types

HTTP::Info
Type

record

ts: time &log

Timestamp for when the request happened.

uid: string &log

Unique ID for the connection.

id: conn_id &log

The connection’s 4-tuple of endpoint addresses/ports.

trans_depth: count &log

Represents the pipelined depth into the connection of this request/response transaction.

method: string &log &optional

Verb used in the HTTP request (GET, POST, HEAD, etc.).

host: string &log &optional

Value of the HOST header.

uri: string &log &optional

URI used in the request.

referrer: string &log &optional

Value of the “referer” header. The comment is deliberately misspelled like the standard declares, but the name used here is “referrer”, spelled correctly.

version: string &log &optional

Value of the version portion of the reply. If you require message-level detail, consider the http_request and http_reply events, which report each message’s version string.

user_agent: string &log &optional

Value of the User-Agent header from the client.

origin: string &log &optional

Value of the Origin header from the client.

request_body_len: count &log &default = 0 &optional

Actual uncompressed content size of the data transferred from the client.

response_body_len: count &log &default = 0 &optional

Actual uncompressed content size of the data transferred from the server.

status_code: count &log &optional

Status code returned by the server.

status_msg: string &log &optional

Status message returned by the server.

info_code: count &log &optional

Last seen 1xx informational reply code returned by the server.

info_msg: string &log &optional

Last seen 1xx informational reply message returned by the server.

tags: set [HTTP::Tags] &log

A set of indicators of various attributes discovered and related to a particular request/response pair.

username: string &log &optional

Username if basic-auth is performed for the request.

password: string &log &optional

Password if basic-auth is performed for the request.

capture_password: bool &default = HTTP::default_capture_password &optional

Determines if the password will be captured for this request.

proxied: set [string] &log &optional

All of the headers that may indicate if the request was proxied.

range_request: bool &default = F &optional

Indicates if this request can assume 206 partial content in response.

orig_fuids: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of file unique IDs. Limited to HTTP::max_files_orig entries.

orig_filenames: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of filenames from the client. Limited to HTTP::max_files_orig entries.

orig_mime_types: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of mime types. Limited to HTTP::max_files_orig entries.

resp_fuids: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of file unique IDs. Limited to HTTP::max_files_resp entries.

resp_filenames: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of filenames from the server. Limited to HTTP::max_files_resp entries.

resp_mime_types: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of mime types. Limited to HTTP::max_files_resp entries.

current_entity: HTTP::Entity &optional

(present if base/protocols/http/entities.zeek is loaded)

The current entity.

orig_mime_depth: count &default = 0 &optional

(present if base/protocols/http/entities.zeek is loaded)

Current number of MIME entities in the HTTP request message body.

resp_mime_depth: count &default = 0 &optional

(present if base/protocols/http/entities.zeek is loaded)

Current number of MIME entities in the HTTP response message body.

client_header_names: vector of string &log &optional

(present if policy/protocols/http/header-names.zeek is loaded)

The vector of HTTP header names sent by the client. No header values are included here, just the header names.

server_header_names: vector of string &log &optional

(present if policy/protocols/http/header-names.zeek is loaded)

The vector of HTTP header names sent by the server. No header values are included here, just the header names.

omniture: bool &default = F &optional

(present if policy/protocols/http/software-browser-plugins.zeek is loaded)

Indicates if the server is an omniture advertising server.

flash_version: string &optional

(present if policy/protocols/http/software-browser-plugins.zeek is loaded)

The unparsed Flash version, if detected.

cookie_vars: vector of string &optional &log

(present if policy/protocols/http/var-extraction-cookies.zeek is loaded)

Variable names extracted from all cookies.

uri_vars: vector of string &optional &log

(present if policy/protocols/http/var-extraction-uri.zeek is loaded)

Variable names from the URI.

The record type which contains the fields of the HTTP log.

HTTP::State
Type

record

pending: table [count] of HTTP::Info

Pending requests.

current_request: count &default = 0 &optional

Current request in the pending queue.

current_response: count &default = 0 &optional

Current response in the pending queue.

trans_depth: count &default = 0 &optional

Track the current deepest transaction. This is meant to cope with missing requests and responses.

Structure to maintain state for an HTTP connection with multiple requests and responses.

HTTP::Tags
Type

enum

HTTP::EMPTY

Placeholder.

HTTP::URI_SQLI

(present if policy/protocols/http/detect-sqli.zeek is loaded)

Indicator of a URI based SQL injection attack.

HTTP::POST_SQLI

(present if policy/protocols/http/detect-sqli.zeek is loaded)

Indicator of client body based SQL injection attack. This is typically the body content of a POST request. Not implemented yet.

HTTP::COOKIE_SQLI

(present if policy/protocols/http/detect-sqli.zeek is loaded)

Indicator of a cookie based SQL injection attack. Not implemented yet.

Indicate a type of attack or compromise in the record to be logged.

Events

HTTP::log_http
Type

event (rec: HTTP::Info)

Event that can be handled to access the HTTP record as it is sent on to the logging framework.

Hooks

HTTP::finalize_http
Type

Conn::RemovalHook

HTTP finalization hook. Remaining HTTP info may get logged when it’s called.

HTTP::log_policy
Type

Log::PolicyHook