urls.zeek¶

Functions for URL handling.

Summary¶

Redefinable Options¶

url_regex: pattern &redef

A regular expression for matching and extracting URLs.

Types¶

URI: record

A URI, as parsed by decompose_uri.

Functions¶

`decompose_uri`: `function`
`find_all_urls`: `function`	Extracts URLs discovered in arbitrary text.
`find_all_urls_without_scheme`: `function`	Extracts URLs discovered in arbitrary text without the URL scheme included.

Detailed Interface¶

Redefinable Options¶

url_regex¶

Type

pattern

Attributes

&redef

Default

/^?(^([a-zA-Z\-]{3,5}):\/\/(-\.)?([^[:blank:]\/?\.#-]+\.?)+(\/[^[:blank:]]*)?)$?/

A regular expression for matching and extracting URLs. This is the @imme_emosol regex from https://mathiasbynens.be/demo/url-regex, adapted for Zeek. It’s not perfect for all of their test cases, but it’s one of the shorter ones that covers most of the test cases.

Types¶

URI¶

Type

record

scheme: string &optional: The URL’s scheme..
netlocation: string: The location, which could be a domain name or an IP address. Left empty if not specified.
portnum: count &optional: Port number, if included in URI.
path: string: Full including the file name. Will be ‘/’ if there’s not path given.
file_name: string &optional: Full file name, including extension, if there is a file name.
file_base: string &optional: The base filename, without extension, if there is a file name.
file_ext: string &optional: The filename’s extension, if there is a file name.
params: table [string] of string &optional: A table of all query parameters, mapping their keys to values, if there’s a query.

A URI, as parsed by decompose_uri.

Functions¶

decompose_uri¶

Type: function (uri: string) : URI

find_all_urls¶

Type: function (s: string) : string_set

Extracts URLs discovered in arbitrary text.

find_all_urls_without_scheme¶

Type: function (s: string) : string_set

Extracts URLs discovered in arbitrary text without the URL scheme included.