main.zeek

HTTP

Implements base functionality for HTTP analysis. The logging model is to log request/response pairs and all relevant metadata together in a single record.

Namespace:: HTTP
Imports:: base/frameworks/tunnels, base/protocols/conn/removal-hooks.zeek, base/utils/files.zeek, base/utils/numbers.zeek

Summary

Runtime Options

`HTTP::default_capture_password`: `bool` `&redef`	This setting changes if passwords used in Basic-Auth are captured or not.
`HTTP::http_methods`: `set` `&redef`	A list of HTTP methods.
`HTTP::max_pending_requests`: `count` `&redef`	Only allow that many pending requests on a single connection.
`HTTP::proxy_headers`: `set` `&redef`	A list of HTTP headers typically used to indicate proxied requests.

Types

`HTTP::Info`: `record`	The record type which contains the fields of the HTTP log.
`HTTP::State`: `record`	Structure to maintain state for an HTTP connection with multiple requests and responses.
`HTTP::Tags`: `enum`	Indicate a type of attack or compromise in the record to be logged.

Redefinitions

`Log::ID`: `enum`	`HTTP::LOG`
`connection`: `record`	New Fields: `connection` http: `HTTP::Info` `&optional` http_state: `HTTP::State` `&optional`
`likely_server_ports`: `set` `&redef`

Events

HTTP::log_http: event

Event that can be handled to access the HTTP record as it is sent on to the logging framework.

Hooks

`HTTP::finalize_http`: `Conn::RemovalHook`	HTTP finalization hook.
`HTTP::log_policy`: `Log::PolicyHook`

Detailed Interface

Runtime Options

HTTP::default_capture_password 

Type:: bool
Attributes:: &redef
Default:: F

This setting changes if passwords used in Basic-Auth are captured or not.

HTTP::http_methods 

Type:

set [string]

Attributes:

&redef

Default:

{
   "POST",
   "PUT",
   "CONNECT",
   "BMOVE",
   "SEARCH",
   "TRACE",
   "LOCK",
   "PROPPATCH",
   "HEAD",
   "OPTIONS",
   "POLL",
   "REPORT",
   "SUBSCRIBE",
   "MOVE",
   "GET",
   "UNLOCK",
   "DELETE",
   "COPY",
   "MKCOL",
   "PROPFIND"
}

A list of HTTP methods. Other methods will generate a weird. Note that the HTTP analyzer will only accept methods consisting solely of letters [A-Za-z].

HTTP::max_pending_requests 

Type:: count
Attributes:: &redef
Default:: 100

Only allow that many pending requests on a single connection. If this number is exceeded, all pending requests are flushed out and request/response tracking reset to prevent unbounded state growth.

HTTP::proxy_headers 

Type:

set [string]

Attributes:

&redef

Default:

{
   "CLIENT-IP",
   "X-FORWARDED-FROM",
   "VIA",
   "XROXY-CONNECTION",
   "PROXY-CONNECTION",
   "X-FORWARDED-FOR",
   "FORWARDED"
}

A list of HTTP headers typically used to indicate proxied requests.

Types

HTTP::Info 

Type:

record

Fields:

ts: time &log: Timestamp for when the request happened.

uid: string &log: Unique ID for the connection.

id: conn_id &log: The connection’s 4-tuple of endpoint addresses/ports.

trans_depth: count &log: Represents the pipelined depth into the connection of this request/response transaction.

method: string &log &optional: Verb used in the HTTP request (GET, POST, HEAD, etc.).

host: string &log &optional: Value of the HOST header.

uri: string &log &optional: URI used in the request.

referrer: string &log &optional: Value of the “referer” header. The comment is deliberately misspelled like the standard declares, but the name used here is “referrer”, spelled correctly.

version: string &log &optional: Value of the version portion of the reply. If you require message-level detail, consider the http_request and http_reply events, which report each message’s version string.

user_agent: string &log &optional: Value of the User-Agent header from the client.

origin: string &log &optional: Value of the Origin header from the client.

request_body_len: count &log &default = 0 &optional: Actual uncompressed content size of the data transferred from the client.

response_body_len: count &log &default = 0 &optional: Actual uncompressed content size of the data transferred from the server.

status_code: count &log &optional: Status code returned by the server.

status_msg: string &log &optional: Status message returned by the server.

info_code: count &log &optional: Last seen 1xx informational reply code returned by the server.

info_msg: string &log &optional: Last seen 1xx informational reply message returned by the server.

tags: set [HTTP::Tags] &log: A set of indicators of various attributes discovered and related to a particular request/response pair.

username: string &log &optional: Username if basic-auth is performed for the request.

password: string &log &optional: Password if basic-auth is performed for the request.

capture_password: bool &default = HTTP::default_capture_password &optional: Determines if the password will be captured for this request.

proxied: set [string] &log &optional: All of the headers that may indicate if the request was proxied.

range_request: bool &default = F &optional: Indicates if this request can assume 206 partial content in response.

orig_fuids: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of file unique IDs. Limited to HTTP::max_files_orig entries.

orig_filenames: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of filenames from the client. Limited to HTTP::max_files_orig entries.

orig_mime_types: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of mime types. Limited to HTTP::max_files_orig entries.

resp_fuids: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of file unique IDs. Limited to HTTP::max_files_resp entries.

resp_filenames: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of filenames from the server. Limited to HTTP::max_files_resp entries.

resp_mime_types: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of mime types. Limited to HTTP::max_files_resp entries.

current_entity: HTTP::Entity &optional

(present if base/protocols/http/entities.zeek is loaded)

The current entity.

orig_mime_depth: count &default = 0 &optional

(present if base/protocols/http/entities.zeek is loaded)

Current number of MIME entities in the HTTP request message body.

resp_mime_depth: count &default = 0 &optional

(present if base/protocols/http/entities.zeek is loaded)

Current number of MIME entities in the HTTP response message body.

client_header_names: vector of string &log &optional

(present if policy/protocols/http/header-names.zeek is loaded)

The vector of HTTP header names sent by the client. No header values are included here, just the header names.

server_header_names: vector of string &log &optional

(present if policy/protocols/http/header-names.zeek is loaded)

The vector of HTTP header names sent by the server. No header values are included here, just the header names.

omniture: bool &default = F &optional

(present if policy/protocols/http/software-browser-plugins.zeek is loaded)

Indicates if the server is an omniture advertising server.

flash_version: string &optional

(present if policy/protocols/http/software-browser-plugins.zeek is loaded)

The unparsed Flash version, if detected.

cookie_vars: vector of string &optional &log

(present if policy/protocols/http/var-extraction-cookies.zeek is loaded)

Variable names extracted from all cookies.

uri_vars: vector of string &optional &log

(present if policy/protocols/http/var-extraction-uri.zeek is loaded)

Variable names from the URI.

The record type which contains the fields of the HTTP log.

HTTP::State 

Type:

record

Fields:

pending: table [count] of HTTP::Info: Pending requests.

current_request: count &default = 0 &optional: Current request in the pending queue.

current_response: count &default = 0 &optional: Current response in the pending queue.

trans_depth: count &default = 0 &optional: Track the current deepest transaction. This is meant to cope with missing requests and responses.