TLS Decryption

Zeek has limited support for decrypting TLS connections, if the necessary key material is available. If decryption is possible, Zeek can forward the decrypted data to other analyzers - like the HTTP analyzer.

Note that this support is currently limited to a single version of TLS and a single cipher suite. Zeek can currently only decrypt TLS 1.2 connections that use the TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 cipher. Any other TLS version or cipher will not be decrypted. We do not currently plan to extend this support to other versions of TLS, or to other ciphersuites.

Capturing and decrypting a trace file

The most common use-case for TLS decryption is to capture a trace file together with the necessary key material to allow Zeek to decrypt it. In principle, it is possible to allow Zeek to decrypt TLS connections in live traffic. However, this kind of setup is much more complex and will require the user to set up a way to transport the key material to Zeek in real-time. We will talk a bit about the possibility of this below.

Capturing a trace file with keys

To be able to decrypt the TLS connections contained in a trace file, we need access to the symmetric keys that were used to encrypt the connection. Specifically, for our supported version of TLS, we need the pre-master secret for the TLS connection.

Firefox and Chrome allow users to capture the key material of all TLS connections that they perform by setting the SSLKEYLOGFILE environment variable. For example, on Mac OS, you can instruct Firefox to record the TLS key material by starting it in the following way:

export SSLKEYLOGFILE=$HOME/keylogfile.txt
open -a firefox

After running Firefox like this, and accessing some web pages, you should end up with the file keylogfile.txt in your home directory that contains lines like this:

# SSL/TLS secrets log file, generated by NSS
CLIENT_RANDOM 47d1becb619e0851ee363c2cf37187228227ca4e680f9a7c0bd15069aa7a5970 ad03ceda4890fa581e989f5e3862023e2a4e3e8ad81325238d908066e1d35cc875979e34c08e6fdfd9d8c6f356e385c1
CLIENT_RANDOM 2095006fcb3f93d255cbb6562587f0dd010212fdee9d233aff64e6ed36cd5c45 0d36faaa2eadbda2a8095f951de1cbac46b81b008fbf391d91951b3485476bab73288a1e17cd0ce80e0fc0401dbe9e3f
CLIENT_RANDOM 8f58b32bf97e7d3856e2fccbbe80798ec2e3f515251082ad63bbc7c231d8bee0 9a7cf946a04718a19f4d20c3f80c1cf8c823c3e2b1c337ef64322d751b410543315f6ecf7dbf45ec9be194a3cc7c1a0f

These log lines contain the pre-master secrets for the connections that your browser established. The secrets are indexed with the client random of the connections. This allows applications (like Zeek) to identify which secret to use to decrypt a connection.

If you capture this key log file together with a trace-file, you will be able to decrypt the sessions using Zeek (assuming they use a supported TLS version and ciphersuite).

Decrypting a trace file

The next step is to convert the keylogfile into a format that can be ingested by the Zeek. This bash-script will perform the conversion:

#!/usr/bin/env bash

if [ $# -ne 1 ]; then
     echo "Script expects one argument (key log filename)" >/dev/stderr
     exit -1


if [ ! -f ${FILE} ]; then
     echo "${FILE} does not exist or is not readable" >/dev/stderr
     exit -1

echo "#fields        client_random   secret"
grep CLIENT_RANDOM ${FILE} | sed 's/^CLIENT_RANDOM ........\(.*\) \(.*\)$/\1 \2/' | sed 's/[A-Za-z0-9][A-Za-z0-9]/\\x&/g'

Note that the script just converts the keylog file in a standard Zeek tsv-file. Furthermore, it removes the first 16 characters of the CLIENT_RANDOM; this is needed due to a design-choice of Zeek that makes accessing the first 8 bytes (equivalent to 16 hex-characters) of the client random inconvenient - thus these bytes are not used for matching.

If you run the bash script on the keylogfile.txt you created earlier, you will get a Zeek tsv-file.

./ ~/keylogfile.txt > ~/keylogfile.log

cat ~/keylogfile.log
#fields      client_random   secret
\x0e\x78\x2d\x35\x63\x95\x5d\x8a\x30\xa9\xcf\xb6\x4f\x47\xf3\x96\x34\x8a\x1e\x79\x1a\xa2\x32\x55\xe2\x2f\xc5\x7a     \x34\x4f\x12\x65\xbf\x43\x40\xb3\x61\x6b\xa0\x16\x5d\x2b\x4d\xb9\xb1\xe8\x4a\x3d\xa2\x42\x0e\x38\xab\x01\x50\x62\x84\xcc\x34\xcd\xe0\x34\x10\xfe\x1a\x02\x30\x49\x74\x6c\x46\x43\xa7\x0c\x67\x0d
\x24\x8c\x7e\x24\xee\xfb\x13\xcd\xee\xde\xb1\xf4\xb6\xd6\xd5\xee\x67\x8d\xd3\xff\xc7\xe7\x39\x23\x18\x3f\x99\xb4     \xe7\xed\x24\x26\x0d\x25\xd9\xfd\xf5\x0f\xc0\xf4\x56\x51\x0e\x4e\xec\x7f\x58\x9c\xaf\x39\x25\x14\x16\xa6\x71\xdd\xea\xfe\xe9\xc0\x93\xbe\x89\x4c\xab\xcc\xff\xb2\xf0\x9a\xea\x98\xf5\xb2\x53\x1e
\x57\xd7\xc7\x7a\x2d\x5e\x35\x29\x2c\xd7\xe7\x94\xee\xf8\x6f\x31\x45\xf6\xbe\x25\x08\xed\x1d\x92\xd2\x0b\x9b\x04     \xc1\x93\x17\x93\xd9\x7d\xd2\x98\xb3\xe0\xdb\x2c\x5d\xbe\x71\x31\xa7\x9a\xf5\x91\xf9\x87\x90\xee\xb7\x79\x9f\x6b\xb4\x1f\x47\xa7\x69\x62\x4b\xa3\x99\x0c\xa9\x43\xf9\xea\x3b\x4d\x5f\x2f\xfe\xfb

Now we can run Zeek on the trace-file that we recorded. We need a small additional script for this, which stops processing while the TLS keylog file is loaded. It also loads the required policy script.

 1@load protocols/ssl/decryption
 2@load base/protocols/http
 4event zeek_init()
 5    {
 6    suspend_processing();
 7    }
 9event Input::end_of_data(name: string, source: string)
10    {
11    if ( name == "tls-keylog-file" )
12        continue_processing();
13    }
$ export ZEEK_TLS_KEYLOG_FILE=~/keylogfile.log
$ zeek -C -r tls/tls-1.2-stream-keylog.pcap tls_decryption-1-suspend-processing.zeek

$ cat conn.log
#separator \x09
#set_separator       ,
#empty_field (empty)
#unset_field -
#path        conn
#open        2022-03-01-16-57-26
#fields      ts      uid     id.orig_h       id.orig_p       id.resp_h       id.resp_p       proto   service duration        orig_bytes      resp_bytes      conn_state      local_orig      local_resp      missed_bytes    history orig_pkts       orig_ip_bytes   resp_pkts       resp_ip_bytes   tunnel_parents
#types       time    string  addr    port    addr    port    enum    string  interval        count   count   string  bool    bool    count   string  count   count   count   count   set[string]
1646150638.631834    CTy5Us4OUaTOcyrPvc   60679   443     tcp     http,ssl        7.246461        10853   151695  SF      -       -       0       ShADadFf        98      15961   139     158931  -
#close       2022-03-01-16-57-26

$ cat http.log
#separator \x09
#set_separator       ,
#empty_field (empty)
#unset_field -
#path        http
#open        2022-03-01-16-57-25
#fields      ts      uid     id.orig_h       id.orig_p       id.resp_h       id.resp_p       trans_depth     method  host    uri     referrer        version user_agent      origin  request_body_len        response_body_len       status_code     status_msg      info_code       info_msg        tags    username        password        proxied orig_fuids      orig_filenames  orig_mime_types resp_fuids      resp_filenames  resp_mime_types
#types       time    string  addr    port    addr    port    count   string  string  string  string  string  string  string  count   count   count   string  count   string  set[enum]       string  string  set[string]     vector[string]  vector[string]  vector[string]  vector[string]  vector[string]  vector[string]
1646150638.735969    CTy5Us4OUaTOcyrPvc   60679   443     1       GET    /assets/akwa/v24/js/akwa.js?.ltc.c61e84978682308f631c   1.1     Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:97.0) Gecko/20100101 Firefox/97.0      -       0       375340  200     OK      -       -       (empty) -       -       -       -       -       -       FSJiWr34wfIujxxtm3      -       text/plain
1646150638.944774    CTy5Us4OUaTOcyrPvc   60679   443     2       GET    /assets/heise/images/mit_technology_review_singleline.b768.ltc.svg   1.1     Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:97.0) Gecko/20100101 Firefox/97.0      -       0       3430    200     OK      -       -       (empty) -       -       -       -       -       -       FgivhC1pvnYeQS4u18      -       text/plain
1646150638.976118    CTy5Us4OUaTOcyrPvc   60679   443     3       GET    /assets/heise/hobell/css/hobell.css?.ltc.3746e7e49abafa23b5fb   1.1     Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:97.0) Gecko/20100101 Firefox/97.0      -       0       85280   200     OK      -       -       (empty) -       -       -       -       -       -       FyvBkl2nwRXf0hkDO1      -       text/plain

Now conn.log shows that the HTTP as well as the SSL analyzers were attached. http.log shows the information from the decrypted HTTP session.

If you try this yourself note that today a lot of encrypted Internet traffic uses HTTP/2. Zeek currently does not ship with an HTTP/2 parser by default. If you capture your own traffic make sure that your browser uses HTTP/1. Alternatively, you can add an HTTP/2 analyzer to Zeek, e.g. using a package.

Decrypting live traffic

In principle, it is possible to decrypt live traffic using this approach. When you want to do this, you have to supply the secrets to Zeek as the connections are happening. Note that there are timing constraints here - the secrets should arrive at the Zeek instance that will decrypt the traffic before encrypted application data is exchanged.

The policy/protocols/ssl/decryption.zeek policy script sets up a two events for this purpose. You can send key material to the Zeek worker in question via Broker, using the /zeek/tls/decryption topic. The two events used for this are SSL::add_keys and SSL::add_secret.

TLS Decryption API

If the policy script does not suit your use-case, you can use the TLS decryption API directly to decrypt a connection. You can use either the set_secret or the set_keys functions to provide the decryption keys for an ongoing SSL connection.

Note that you will have to make sure to set SSL::disable_analyzer_after_detection to false if you use this functionality directly.