main.zeek

HTTP

Implements base functionality for HTTP analysis. The logging model is to log request/response pairs and all relevant metadata together in a single record.

Namespace:: HTTP
Imports:: base/frameworks/tunnels, base/protocols/conn/removal-hooks.zeek, base/utils/files.zeek, base/utils/numbers.zeek

Summary

Runtime Options

`HTTP::default_capture_password`: `bool` `&redef`	This setting changes if passwords used in Basic-Auth are captured or not.
`HTTP::http_methods`: `set` `&redef`	A list of HTTP methods.
`HTTP::max_pending_requests`: `count` `&redef`	Only allow that many pending requests on a single connection.
`HTTP::proxy_headers`: `set` `&redef`	A list of HTTP headers typically used to indicate proxied requests.

Redefinable Options

`HTTP::default_max_field_string_bytes`: `count` `&redef`	The maximum number of bytes that a single string field can contain when logging.
`HTTP::ports`: `set` `&redef`	Well-known ports for HTTP.

Types

`HTTP::Info`: `record`	The record type which contains the fields of the HTTP log.
`HTTP::State`: `record`	Structure to maintain state for an HTTP connection with multiple requests and responses.
`HTTP::Tags`: `enum`	Indicate a type of attack or compromise in the record to be logged.

Redefinitions

`Log::ID`: `enum`	`HTTP::LOG`
`connection`: `record`	New Fields: `connection` http: `HTTP::Info` `&optional` http_state: `HTTP::State` `&optional`

Events

HTTP::log_http: event

Event that can be handled to access the HTTP record as it is sent on to the logging framework.

Hooks

`HTTP::finalize_http`: `Conn::RemovalHook`	HTTP finalization hook.
`HTTP::log_policy`: `Log::PolicyHook`

Detailed Interface

Runtime Options

HTTP::default_capture_password 

Type:: bool
Attributes:: &redef
Default:: F

This setting changes if passwords used in Basic-Auth are captured or not.

HTTP::http_methods 

Type:

set [string]

Attributes:

&redef

Default:

{
   "POST",
   "PUT",
   "CONNECT",
   "BMOVE",
   "SEARCH",
   "TRACE",
   "LOCK",
   "PROPPATCH",
   "HEAD",
   "OPTIONS",
   "POLL",
   "REPORT",
   "SUBSCRIBE",
   "MOVE",
   "GET",
   "UNLOCK",
   "DELETE",
   "COPY",
   "MKCOL",
   "PROPFIND"
}

A list of HTTP methods. Other methods will generate a weird. Note that the HTTP analyzer will only accept methods consisting solely of letters [A-Za-z].

HTTP::max_pending_requests 

Type:: count
Attributes:: &redef
Default:: 100

Only allow that many pending requests on a single connection. If this number is exceeded, all pending requests are flushed out and request/response tracking reset to prevent unbounded state growth.

HTTP::proxy_headers 

Type:

set [string]

Attributes:

&redef

Default:

{
   "CLIENT-IP",
   "X-FORWARDED-FROM",
   "VIA",
   "XROXY-CONNECTION",
   "PROXY-CONNECTION",
   "X-FORWARDED-FOR",
   "FORWARDED"
}

A list of HTTP headers typically used to indicate proxied requests.

Redefinable Options

HTTP::default_max_field_string_bytes 

Type:: count
Attributes:: &redef
Default:: 0

The maximum number of bytes that a single string field can contain when logging. If a string reaches this limit, the log output for the field will be truncated. Setting this to zero disables the limiting. HTTP has no maximum length for various fields such as the URI, so this is set to zero by default.

HTTP::ports 

Type:

set [port]

Attributes:

&redef

Default:

{
   80/tcp,
   8888/tcp,
   81/tcp,
   8000/tcp,
   3128/tcp,
   8080/tcp,
   631/tcp,
   1080/tcp
}

Well-known ports for HTTP.

Types

HTTP::Info 

Type:

record

Fields:

ts: time &log: Timestamp for when the request happened.

uid: string &log: Unique ID for the connection.

id: conn_id &log: The connection’s 4-tuple of endpoint addresses/ports.

trans_depth: count &log: Represents the pipelined depth into the connection of this request/response transaction.

method: string &log &optional: Verb used in the HTTP request (GET, POST, HEAD, etc.).

host: string &log &optional: Value of the HOST header.

uri: string &log &optional: URI used in the request.

referrer: string &log &optional: Value of the “referer” header. The comment is deliberately misspelled like the standard declares, but the name used here is “referrer”, spelled correctly.

version: string &log &optional: Value of the version portion of the reply. If you require message-level detail, consider the http_request and http_reply events, which report each message’s version string.

user_agent: string &log &optional: Value of the User-Agent header from the client.

origin: string &log &optional: Value of the Origin header from the client.

request_body_len: count &log &default = 0 &optional: Actual uncompressed content size of the data transferred from the client.

response_body_len: count &log &default = 0 &optional: Actual uncompressed content size of the data transferred from the server.

status_code: count &log &optional: Status code returned by the server.

status_msg: string &log &optional: Status message returned by the server.

info_code: count &log &optional: Last seen 1xx informational reply code returned by the server.

info_msg: string &log &optional: Last seen 1xx informational reply message returned by the server.

tags: set [HTTP::Tags] &log: A set of indicators of various attributes discovered and related to a particular request/response pair.

username: string &log &optional: Username if basic-auth is performed for the request.

password: string &log &optional: Password if basic-auth is performed for the request.

capture_password: bool &default = HTTP::default_capture_password &optional: Determines if the password will be captured for this request.

proxied: set [string] &log &optional: All of the headers that may indicate if the request was proxied.

range_request: bool &default = F &optional: Indicates if this request can assume 206 partial content in response.

orig_fuids: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of file unique IDs. Limited to HTTP::max_files_orig entries.

orig_filenames: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of filenames from the client. Limited to HTTP::max_files_orig entries.

orig_mime_types: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of mime types. Limited to HTTP::max_files_orig entries.

resp_fuids: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of file unique IDs. Limited to HTTP::max_files_resp entries.

resp_filenames: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of filenames from the server. Limited to HTTP::max_files_resp entries.

resp_mime_types: vector of string &log &optional

(present if base/protocols/http/entities.zeek is loaded)

An ordered vector of mime types. Limited to HTTP::max_files_resp entries.

current_entity: HTTP::Entity &optional

(present if base/protocols/http/entities.zeek is loaded)

The current entity.

orig_mime_depth: count &default = 0 &optional

(present if base/protocols/http/entities.zeek is loaded)

Current number of MIME entities in the HTTP request message body.

resp_mime_depth: count &default = 0 &optional

(present if base/protocols/http/entities.zeek is loaded)

Current number of MIME entities in the HTTP response message body.

client_header_names: vector of string &log &optional

(present if policy/protocols/http/header-names.zeek is loaded)

The vector of HTTP header names sent by the client. No header values are included here, just the header names.

server_header_names: vector of string &log &optional

(present if policy/protocols/http/header-names.zeek is loaded)

The vector of HTTP header names sent by the server. No header values are included here, just the header names.

omniture: bool &default = F &optional

(present if policy/protocols/http/software-browser-plugins.zeek is loaded)

Indicates if the server is an omniture advertising server.

flash_version: string &optional

(present if policy/protocols/http/software-browser-plugins.zeek is loaded)

The unparsed Flash version, if detected.

cookie_vars: vector of string &optional &log

(present if policy/protocols/http/var-extraction-cookies.zeek is loaded)

Variable names extracted from all cookies.

uri_vars: vector of string &optional &log

(present if policy/protocols/http/var-extraction-uri.zeek is loaded)

Variable names from the URI.

The record type which contains the fields of the HTTP log.

HTTP::State 

Type:

record

Fields:

pending: table [count] of HTTP::Info: Pending requests.

current_request: count &default = 0 &optional: Current request in the pending queue.

current_response: count &default = 0 &optional: Current response in the pending queue.

trans_depth: count &default = 0 &optional: Track the current deepest transaction. This is meant to cope with missing requests and responses.