ssl.log

In the section discussing the http.log, we noted that most HTTP traffic is now encrypted and transmitted as HTTPS. Zeek does not create a https.log, because Zeek (or other network inspection tools, for that matter) does not natively recognize HTTP when it is encrypted as HTTPS.

HTTPS is most often encrypted using Transport Layer Security (TLS), which presents many variants in live traffic. Zeek parses TLS traffic and records its findings in the ssl.log. SSL refers to Secure Sockets Layer, an obsolete predecessor to TLS.

TLS is not restricted to encrypting HTTPS, however. Many other protocols use TLS to encrypt their contents, including Simple Mail Transfer Protocol (SMTP).

Remember that to see the meaning of each field in the ssl.log, check SSL::Info.

Reviewing TLS Versions Seen on the Network

To get an idea of the sorts of TLS traffic running in my network, I ran the following command to search hundreds of days of Zeek ssl.log entries:

$ for i in `find . -name ssl*.log.gz`; do zcat $i; done | jq '[."version"]' | grep -v "\]" | grep -v "\[" | sort -n | uniq -c | sort -rn

11279341   "TLSv12"
2877117   "TLSv13"
 303084   "unknown-64282"
 198154   null
  23181   "TLSv10"
   5756   "TLSv11"
    348   "DTLSv12"
     78   "DTLSv10"

TLS 1.0 and 1.1 are obsolete. TLS 1.2 and 1.3 are common, with 1.3 gaining ground on 1.2 DTLS is a variant used to encrypt UDP traffic. unknown-64282 is apparently a Facebook-created variant of TLS 1.3. Almost 20,000 connections advertised no TLS version, but were recognized by Zeek as some form of TLS.

To try to see what protocols the TLS might be encrypting, I ran the following command to search 10 days of Zeek ssl.log entries:

$ for i in `find ./2020-08-1* -name ssl*.log.gz`; do zcat $i; done | jq -c '[."version", ."next_protocol"]' | sort -n | uniq -c | sort -rn

246868 ["TLSv12",null]
144291 ["TLSv13",null]
 86708 ["TLSv12","http/1.1"]
 85082 ["TLSv12","h2"]
  8450 ["unknown-64282",null]
  1966 [null,null]
   722 ["TLSv12","apns-pack-v1:4096:4096"]
   504 ["TLSv10",null]
   234 ["TLSv10","http/1.1"]
   154 ["TLSv12","grpc-exp"]
    83 ["TLSv11",null]
    13 ["DTLSv12",null]

HTTP/1.1 is obviously HTTP. The h2 entry refers to the newer HTTP/2 protocol. The apns-pack-v1:4096:4096 entry appears to refer to Apple Push Notification Service, which utilizes Application Layer Protocol Negotiation (ALPN), a TLS extension. The grpc-exp entry appears to refer to another ALPN method that uses the gRPC remote procedure call (RPC) library.

With this brief look at the types of TLS traffic one might find in a network done, it’s time to look at a sample connection that generates a ssl.log entry.

Preparing to Inspect the ssl.log

To generate network traffic that uses TLS encryption, I retrieved the index page of the https://www.taosecurity.com using curl.

After processing the traffic with Zeek, I had several log files to analyze. First let’s look at the conn.log. We will focus on the Web session itself, and not related traffic like any DNS lookups required to resolve the hostname to an IP address.

{
  "ts": 1598377391.716515,
  "uid": "CsukF91Bx9mrqdEaH9",
  "id.orig_h": "192.168.4.49",
  "id.orig_p": 56718,
  "id.resp_h": "13.32.202.10",
  "id.resp_p": 443,
  "proto": "tcp",
  "service": "ssl",
  "duration": 0.497269868850708,
  "orig_bytes": 929,
  "resp_bytes": 31113,
  "conn_state": "SF",
  "missed_bytes": 0,
  "history": "ShADadfF",
  "orig_pkts": 37,
  "orig_ip_bytes": 2861,
  "resp_pkts": 34,
  "resp_ip_bytes": 32889
}

We have a client, 192.168.4.49, interacting with a server, 13.32.202.10, offering an encrypted service on port 443 TCP. Zeek reports this as ssl, but that is a generic term that applies to TLS as well. We can use the connection identifier, CsukF91Bx9mrqdEaH9, to find associated Zeek logs.

Inspecting the ssl.log When TLS 1.2 Applies

Using the connection identifier, we find the associated ssl.log entry for this conversation.

{
  "ts": 1598377391.921726,
  "uid": "CsukF91Bx9mrqdEaH9",
  "id.orig_h": "192.168.4.49",
  "id.orig_p": 56718,
  "id.resp_h": "13.32.202.10",
  "id.resp_p": 443,
  "version": "TLSv12",
  "cipher": "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
  "curve": "secp256r1",
  "server_name": "www.taosecurity.com",
  "resumed": false,
  "next_protocol": "h2",
  "established": true,
  "cert_chain_fuids": [
    "F2XEvj1CahhdhtfvT4",
    "FZ7ygD3ERPfEVVohG9",
    "F7vklpOKI4yX9wmvh",
    "FAnbnR32nIIr2j9XV"
  ],
  "client_cert_chain_fuids": [],
  "subject": "CN=www.taosecurity.com",
  "issuer": "CN=Amazon,OU=Server CA 1B,O=Amazon,C=US"
}

This is a rich log entry that tells us a lot about the connection. We see that the server and client agree to speak TLS 1.2, with the designated cipher suite and elliptic curve. The server name, www.taosecurity.com appears, which matches the subject of the certificate presented by the Web server. We can see that Amazon issued the certificate. The next protocol involved was HTTP/2, as the next_protocol field lists h2. Zeek provides file identifiers for the four certificates that the server presented to the client. The client did not present any certificates to the server.

We will use the certificate information when we look at the next log in our series, the x509.log.

Inspecting the ssl.log When TLS 1.3 Applies

The last section showed Zeek’s ssl.log when visiting a server that negotiated a TLS 1.2 connection. The following example shows how the situation changes when the parties use TLS 1.3.

To generate the traffic, I used curl with a switch to try TLS 1.3 encryption.

$ curl -v --tlsv1.3 https://www.taosecurity.com

curl provided the following, in addition to the content of the Web site:

* Connected to www.taosecurity.com (13.32.202.2) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: C:\ProgramData\chocolatey\lib\curl\tools\curl-7.72.0-win64-mingw\bin\curl-ca-bundle.crt
  CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [19 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4880 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=www.taosecurity.com
*  start date: Jun  1 00:00:00 2020 GMT
*  expire date: Jul  1 12:00:00 2021 GMT
*  subjectAltName: host "www.taosecurity.com" matched cert's "www.taosecurity.com"
*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x1f9ff0c7600)
} [5 bytes data]
> GET / HTTP/2
> Host: www.taosecurity.com
> user-agent: curl/7.72.0
> accept: */*
>
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
} [5 bytes data]
< HTTP/2 200
< content-type: text/html
< content-length: 28708
< date: Tue, 01 Sep 2020 18:07:59 GMT
< last-modified: Tue, 01 Sep 2020 14:36:01 GMT
< etag: "9a6a530f507d79ba54daa5872b3cad22"
< accept-ranges: bytes
< server: AmazonS3
< vary: Accept-Encoding
< x-cache: Miss from cloudfront
< via: 1.1 c09a013ad199e52fd50ddc5543a72f45.cloudfront.net (CloudFront)
< x-amz-cf-pop: IAD66-C1
< x-amz-cf-id: wXc1bcKla5qIePZ29LBk1fgATzgf1jLYiRvSmnyZcb7Q1eB_ZJSbaA==
<
{ [16032 bytes data]

Note that the certificate details are visible here, because we are looking from the perspective of the Web client, not a passive network observation system.

Here is the conn.log:

{
  "ts": 1598983678.546522,
  "uid": "CcJfBs3hXLJn7oHVu7",
  "id.orig_h": "192.168.4.142",
  "id.orig_p": 58802,
  "id.resp_h": "13.32.202.2",
  "id.resp_p": 443,
  "proto": "tcp",
  "service": "ssl",
  "duration": 0.13053107261657715,
  "orig_bytes": 831,
  "resp_bytes": 34650,
  "conn_state": "SF",
  "missed_bytes": 0,
  "history": "ShADadFf",
  "orig_pkts": 17,
  "orig_ip_bytes": 1523,
  "resp_pkts": 30,
  "resp_ip_bytes": 35862
}

Here is the ssl.log:

{
  "ts": 1598983678.585087,
  "uid": "CcJfBs3hXLJn7oHVu7",
  "id.orig_h": "192.168.4.142",
  "id.orig_p": 58802,
  "id.resp_h": "13.32.202.2",
  "id.resp_p": 443,
  "version": "TLSv13",
  "cipher": "TLS_AES_128_GCM_SHA256",
  "curve": "x25519",
  "server_name": "www.taosecurity.com",
  "resumed": true,
  "established": true
}

Note that there is no mention of certificates in the ssl.log. TLS 1.3 hides these from passive observation systems. We are able to see the server name, www.taosecurity.com, however, as well as some information about the encryption used. These include the TLS version, the cipher, and the elliptic curve.

Inspecting the ssl.log When ESNI/ECH Applies

There is one more concern for an analyst working with the ssl.log.

Encrypted Server Name Indication (ESNI) or Encrypted Client Hello (ECH) are methods by which the Server Name Identification field is no longer sent as plain text. The mechanics of this process are less important than the effects on Zeek ssl.log entries.

To generate traffic for this example, I used a modern version of Firefox, configured to support ESNI, and visited a Web site, https://only.esni.defo.ie/, that only accepts connections from systems supporting ESNI.

After processing the traffic with Zeek, I had the following logs.

First, I had two conn.log entries:

{"ts":1598631659.652789,"uid":"Cg9oVc87cdxWf5Dla","id.orig_h":"192.168.4.142","id.orig_p":63213,"id.resp_h":"185.24.233.103","id.resp_p":443,"proto":"tcp","service":"ssl","duration":5.702061891555786,"orig_bytes":1467,"resp_bytes":3160,"conn_state":"SF","missed_bytes":0,"history":"ShADadTtFf","orig_pkts":11,"orig_ip_bytes":2347,"resp_pkts":8,"resp_ip_bytes":4645}

{"ts":1598631659.331871,"uid":"Cixuvq2LQrbqxU4Y17","id.orig_h":"192.168.4.142","id.orig_p":63210,"id.resp_h":"185.24.233.103","id.resp_p":443,"proto":"tcp","service":"ssl","duration":6.023154020309448,"orig_bytes":2193,"resp_bytes":45269,"conn_state":"SF","missed_bytes":0,"history":"ShADadFf","orig_pkts":14,"orig_ip_bytes":2765,"resp_pkts":37,"resp_ip_bytes":46761}

Second, I had two ssl.log entries:

{
  "ts": 1598631659.431907,
  "uid": "Cixuvq2LQrbqxU4Y17",
  "id.orig_h": "192.168.4.142",
  "id.orig_p": 63210,
  "id.resp_h": "185.24.233.103",
  "id.resp_p": 443,
  "version": "TLSv13",
  "cipher": "TLS_AES_256_GCM_SHA384",
  "curve": "x25519",
  "resumed": true,
  "established": true
}
{
  "ts": 1598631659.752715,
  "uid": "Cg9oVc87cdxWf5Dla",
  "id.orig_h": "192.168.4.142",
  "id.orig_p": 63213,
  "id.resp_h": "185.24.233.103",
  "id.resp_p": 443,
  "version": "TLSv13",
  "cipher": "TLS_AES_256_GCM_SHA384",
  "curve": "x25519",
  "resumed": true,
  "established": true
}

As you can see, there is no identifying information in the ssl.log here. There are no certificate identifier entries either, although we will talk about that log type in the next section. As the visit to https://only.esni.defo.ie/ also used DNS over HTTPs (DoH), there is no DNS record showing the identity of the remote server, as might be revealed in a conventional DNS request and response.

As you might expect, this situation has some network security monitoring practitioners concerned by the loss of visibility, and the opportunity for intruders to leverage ESNI-enabled servers and Doh-enabled clients to evade inspection.

Leveraging JA3 and JA3S

JA3 and JA3S are mechanisms to profile the TLS implementations on clients and servers, respectively. These are clever tools to tell analysts more about each end of a connection. To learn more, see the following project page:

https://github.com/salesforce/ja3

When running Zeek with the JA3 and JA3S packages, the scripts will append data to the ssl.log as follows.

In the first example, a Web client (curl) connects to the Google Web site using TLS 1.3. The ssl.log shows the following entry.

{
  "ts": "2020-09-16T14:01:26.194646Z",
  "uid": "CH3QeG4kCxFL8eZrs1",
  "id.orig_h": "192.168.4.37",
  "id.orig_p": 58842,
  "id.resp_h": "172.217.15.100",
  "id.resp_p": 443,
  "version": "TLSv13",
  "cipher": "TLS_AES_256_GCM_SHA384",
  "curve": "x25519",
  "server_name": "www.google.com",
  "resumed": true,
  "established": true,
  "ja3": "3830b2a4fbcea64e74db382e467f5b3b",
  "ja3s": "907bf3ecef1c987c889946b737b43de8"
}

Zeek computes the JA3 (client) and JA3S (server) hashes as shown.

In the second example, the same Web client connects to the Corelight Web site.

{
  "ts": "2020-09-16T13:58:21.878466Z",
  "uid": "CtbyI4sDwTIPROUv6",
  "id.orig_h": "192.168.4.37",
  "id.orig_p": 49572,
  "id.resp_h": "99.86.230.78",
  "id.resp_p": 443,
  "version": "TLSv13",
  "cipher": "TLS_AES_128_GCM_SHA256",
  "curve": "x25519",
  "server_name": "www.corelight.com",
  "resumed": true,
  "established": true,
  "ja3": "3830b2a4fbcea64e74db382e467f5b3b",
  "ja3s": "f4febc55ea12b31ae17cfb7e614afda8"
}

The JA3 (client) hash has stayed the same, but the JA3S (server) hash has changed.

In the third example, the same Web client connects to the TaoSecurity Web site.

{
  "ts": "2020-09-16T13:54:57.033503Z",
  "uid": "CXc63QyS40XspAmcd",
  "id.orig_h": "192.168.4.37",
  "id.orig_p": 41608,
  "id.resp_h": "99.84.222.6",
  "id.resp_p": 443,
  "version": "TLSv13",
  "cipher": "TLS_AES_128_GCM_SHA256",
  "curve": "x25519",
  "server_name": "www.taosecurity.com",
  "resumed": true,
  "established": true,
  "ja3": "0bae189478c11bed9d6259ae0ffc9493",
  "ja3s": "f4febc55ea12b31ae17cfb7e614afda8"
}

This is an odd result. The JA3 (client) hash has changed, but the JA3S (server) hash has stayed the same. I can explain the server hash staying the same by noting that both the Corelight and TaoSecurity Web sites appear to be hosted by Amazon, meaning the Web servers providing each site are offering the same TLS parameters.

However, I would have expected the JA3 (client) hash to have been the same as the previous two examples. I repeated the connection and got the same JA3 and JA3S hashes.

Conclusion

This section showed that the default ssl.log provides several details of interest to defenders, even when inspecting encrypted traffic. As administrators and intruders deploy newer encryption technologies, however, defenders will find it increasingly difficult to differentiate among normal, suspicious, and malicious traffic.