TLS Decryption
Zeek has limited support for decrypting TLS connections, if the necessary key material is available. If decryption is possible, Zeek can forward the decrypted data to other analyzers - like the HTTP analyzer.
Note that this support is currently limited to a single version of TLS and a single cipher suite. Zeek can currently only decrypt TLS 1.2 connections that use the TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 cipher. Any other TLS version or cipher will not be decrypted. We do not currently plan to extend this support to other versions of TLS, or to other ciphersuites.
Capturing and decrypting a trace file
The most common use-case for TLS decryption is to capture a trace file together with the necessary key material to allow Zeek to decrypt it. In principle, it is possible to allow Zeek to decrypt TLS connections in live traffic. However, this kind of setup is much more complex and will require the user to set up a way to transport the key material to Zeek in real-time. We will talk a bit about the possibility of this below.
Capturing a trace file with keys
To be able to decrypt the TLS connections contained in a trace file, we need access to the symmetric keys that were used to encrypt the connection. Specifically, for our supported version of TLS, we need the pre-master secret for the TLS connection.
Firefox and Chrome allow users to capture the key material of all TLS connections that they perform by setting
the SSLKEYLOGFILE environment variable. For example, on Mac OS, you can instruct Firefox to record the TLS
key material by starting it in the following way:
export SSLKEYLOGFILE=$HOME/keylogfile.txt
open -a firefox
After running Firefox like this, and accessing some web pages, you should end up with the file keylogfile.txt
in your home directory that contains lines like this:
# SSL/TLS secrets log file, generated by NSS
CLIENT_RANDOM 47d1becb619e0851ee363c2cf37187228227ca4e680f9a7c0bd15069aa7a5970 ad03ceda4890fa581e989f5e3862023e2a4e3e8ad81325238d908066e1d35cc875979e34c08e6fdfd9d8c6f356e385c1
CLIENT_RANDOM 2095006fcb3f93d255cbb6562587f0dd010212fdee9d233aff64e6ed36cd5c45 0d36faaa2eadbda2a8095f951de1cbac46b81b008fbf391d91951b3485476bab73288a1e17cd0ce80e0fc0401dbe9e3f
CLIENT_RANDOM 8f58b32bf97e7d3856e2fccbbe80798ec2e3f515251082ad63bbc7c231d8bee0 9a7cf946a04718a19f4d20c3f80c1cf8c823c3e2b1c337ef64322d751b410543315f6ecf7dbf45ec9be194a3cc7c1a0f
These log lines contain the pre-master secrets for the connections that your browser established. The secrets are indexed with the client random of the connections. This allows applications (like Zeek) to identify which secret to use to decrypt a connection.
If you capture this key log file together with a trace-file, you will be able to decrypt the sessions using Zeek (assuming they use a supported TLS version and ciphersuite).
Decrypting a trace file
The next step is to convert the keylogfile into a format that can be ingested by the Zeek. This bash-script will perform the conversion:
#!/usr/bin/env bash
if [ $# -ne 1 ]; then
echo "Script expects one argument (key log filename)" >/dev/stderr
exit -1
fi
FILE=$1
if [ ! -f ${FILE} ]; then
echo "${FILE} does not exist or is not readable" >/dev/stderr
exit -1
fi
echo "#fields client_random secret"
grep CLIENT_RANDOM ${FILE} | sed 's/^CLIENT_RANDOM ........\(.*\) \(.*\)$/\1 \2/' | sed 's/[A-Za-z0-9][A-Za-z0-9]/\\x&/g'
Note that the script just converts the keylog file in a standard Zeek tsv-file. Furthermore, it removes the first 16 characters of the CLIENT_RANDOM; this is needed due to a design-choice of Zeek that makes accessing the first 8 bytes (equivalent to 16 hex-characters) of the client random inconvenient - thus these bytes are not used for matching.
If you run the bash script on the keylogfile.txt you created earlier, you will get a Zeek tsv-file.
./convert-keylog.sh ~/keylogfile.txt > ~/keylogfile.log
cat ~/keylogfile.log
#fields client_random secret
\x0e\x78\x2d\x35\x63\x95\x5d\x8a\x30\xa9\xcf\xb6\x4f\x47\xf3\x96\x34\x8a\x1e\x79\x1a\xa2\x32\x55\xe2\x2f\xc5\x7a \x34\x4f\x12\x65\xbf\x43\x40\xb3\x61\x6b\xa0\x16\x5d\x2b\x4d\xb9\xb1\xe8\x4a\x3d\xa2\x42\x0e\x38\xab\x01\x50\x62\x84\xcc\x34\xcd\xe0\x34\x10\xfe\x1a\x02\x30\x49\x74\x6c\x46\x43\xa7\x0c\x67\x0d
\x24\x8c\x7e\x24\xee\xfb\x13\xcd\xee\xde\xb1\xf4\xb6\xd6\xd5\xee\x67\x8d\xd3\xff\xc7\xe7\x39\x23\x18\x3f\x99\xb4 \xe7\xed\x24\x26\x0d\x25\xd9\xfd\xf5\x0f\xc0\xf4\x56\x51\x0e\x4e\xec\x7f\x58\x9c\xaf\x39\x25\x14\x16\xa6\x71\xdd\xea\xfe\xe9\xc0\x93\xbe\x89\x4c\xab\xcc\xff\xb2\xf0\x9a\xea\x98\xf5\xb2\x53\x1e
\x57\xd7\xc7\x7a\x2d\x5e\x35\x29\x2c\xd7\xe7\x94\xee\xf8\x6f\x31\x45\xf6\xbe\x25\x08\xed\x1d\x92\xd2\x0b\x9b\x04 \xc1\x93\x17\x93\xd9\x7d\xd2\x98\xb3\xe0\xdb\x2c\x5d\xbe\x71\x31\xa7\x9a\xf5\x91\xf9\x87\x90\xee\xb7\x79\x9f\x6b\xb4\x1f\x47\xa7\x69\x62\x4b\xa3\x99\x0c\xa9\x43\xf9\xea\x3b\x4d\x5f\x2f\xfe\xfb
Now we can run Zeek on the trace-file that we recorded. We need a small additional script for this, which stops processing while the TLS keylog file is loaded. It also loads the required policy script.
1@load protocols/ssl/decryption
2@load base/protocols/http
3
4event zeek_init()
5 {
6 suspend_processing();
7 }
8
9event Input::end_of_data(name: string, source: string)
10 {
11 if ( name == "tls-keylog-file" )
12 continue_processing();
13 }
$ export ZEEK_TLS_KEYLOG_FILE=~/keylogfile.log
$ zeek -C -r tls/tls-1.2-stream-keylog.pcap tls_decryption-1-suspend-processing.zeek
$ cat conn.log
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path conn
#open 2022-03-01-16-57-26
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents
#types time string addr port addr port enum string interval count count string bool bool count string count count count count set[string]
1646150638.631834 CTy5Us4OUaTOcyrPvc 192.168.20.12 60679 193.99.144.85 443 tcp http,ssl 7.246461 10853 151695 SF - - 0 ShADadFf 98 15961 139 158931 -
#close 2022-03-01-16-57-26
$ cat http.log
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path http
#open 2022-03-01-16-57-25
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer version user_agent origin request_body_len response_body_len status_code status_msg info_code info_msg tags username password proxied orig_fuids orig_filenames orig_mime_types resp_fuids resp_filenames resp_mime_types
#types time string addr port addr port count string string string string string string string count count count string count string set[enum] string string set[string] vector[string] vector[string] vector[string] vector[string] vector[string] vector[string]
1646150638.735969 CTy5Us4OUaTOcyrPvc 192.168.20.12 60679 193.99.144.85 443 1 GET www.heise.de /assets/akwa/v24/js/akwa.js?.ltc.c61e84978682308f631c https://www.heise.de/ 1.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:97.0) Gecko/20100101 Firefox/97.0 - 0 375340 200 OK - - (empty) - - - - - - FSJiWr34wfIujxxtm3 - text/plain
1646150638.944774 CTy5Us4OUaTOcyrPvc 192.168.20.12 60679 193.99.144.85 443 2 GET www.heise.de /assets/heise/images/mit_technology_review_singleline.b768.ltc.svg https://www.heise.de/ 1.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:97.0) Gecko/20100101 Firefox/97.0 - 0 3430 200 OK - - (empty) - - - - - - FgivhC1pvnYeQS4u18 - text/plain
1646150638.976118 CTy5Us4OUaTOcyrPvc 192.168.20.12 60679 193.99.144.85 443 3 GET www.heise.de /assets/heise/hobell/css/hobell.css?.ltc.3746e7e49abafa23b5fb https://www.heise.de/ 1.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:97.0) Gecko/20100101 Firefox/97.0 - 0 85280 200 OK - - (empty) - - - - - - FyvBkl2nwRXf0hkDO1 - text/plain
...
Now conn.log shows that the HTTP as well as the SSL analyzers were attached. http.log shows the
information from the decrypted HTTP session.
If you try this yourself note that today a lot of encrypted Internet traffic uses HTTP/2. Zeek currently does not ship with an HTTP/2 parser by default. If you capture your own traffic make sure that your browser uses HTTP/1. Alternatively, you can add an HTTP/2 analyzer to Zeek, e.g. using a package.
Decrypting live traffic
In principle, it is possible to decrypt live traffic using this approach. When you want to do this, you have to supply the secrets to Zeek as the connections are happening. Note that there are timing constraints here - the secrets should arrive at the Zeek instance that will decrypt the traffic before encrypted application data is exchanged.
The policy/protocols/ssl/decryption.zeek policy script sets up a two events for this purpose. You can send key material
to the Zeek worker in question via Broker, using the /zeek/tls/decryption topic. The two events used for this are
SSL::add_keys and SSL::add_secret.
TLS Decryption API
If the policy script does not suit your use-case, you can use the TLS decryption API directly to decrypt a connection. You can use either the
set_secret or the set_keys functions to provide the decryption keys for an ongoing SSL connection.
Note that you will have to make sure to set SSL::disable_analyzer_after_detection to false if you use this functionality directly.