SharkFest ’17 Europe
SSL/TLS Decryption
uncovering secrets
Wednesday November 8th, 2017
Peter Wu
Wireshark Core Developer
peter@[Link]
1
About me
I Wireshark contributor since 2013, core developer since 2015.
I Areas of interest: TLS, Lua, security, . . .
I Developed a VoIP product based on WebRTC.
I Cloudflare crypto intern.
2
Secrets
I Things that people care about: pictures,
videos, documents, email conversations,
passwords, . . .
I Application Data: cookies, API keys, Request
URI, User Agent, form data, response body, . . .
I How to keep these safe when sending it over
the internet or over your local Wi-Fi network?
3
Transport Layer Security (TLS)
I Provides secure communication channel between two endpoints (client and server).
I Network protocol with two components:
I Handshake Protocol: exchange capabilities, establish trust and establish keys.
I Record Protocol: carries messages and protects application data fragments.
Application Application
TLS TLS TLS
TCP TCP TCP TCP
IP IP IP IP
... ... ... ...
Client Server
4
Secure Sockets Layer (SSL) versus Transport Layer Security (TLS)
I SSLv3: old (RFC 6101, 1996) and deprecated (RFC 7568, 2015). Do not use it!
I TLS 1.0 (RFC 2246, 1999), 1.1 (RFC 4346, 2006), 1.2 (RFC 5246, 2008).
I Changes:
I New versions are generally fixing weaknesses due to new attacks.
I TLS 1.0 (RFC 3546, 2003) and up allow for extensions, like Server Name Indication
(SNI) to support virtual hosts.
I TLS 1.2: new authenticated encryption with additional data (AEAD) mode.
I “SSL” term still stuck, e.g. “SSL certificate”, “SSL library” and field names in
Wireshark (e.g. [Link] type).
I Mail protocols: TLS often refers to STARTTLS while SSL directly starts with the
handshake.
5
“Secure” communication channel
I Symmetric-key algorithms: encrypt/decrypt bulk (application) data using a single
(secret) symmetric key. Examples: AES, 3DES, RC4.
I How to create such a secret? For example, AES-256 needs a 256-bit key.
I Public-key cryptography: a (secret) private key and a related public key.
I Mathematically hard to compute private key from public key.
I Encrypt data with public key, decrypt with private key.
I Limitation: maximum data size for RSA is equal to modulus size, 2048-4096 bits.
I Idea: generate a random premaster secret and encrypt it with the public RSA key.
I Where to retrieve this RSA public key from?
6
Certificates and trust
I Public key is embedded in a X.509 certificate.
I How can this certificate be trusted?
I A Certificate Authority (issuer) signs the
certificate with its private key.
I Public-key cryptography: use a private (secret)
key and a public key with small data.
I Compress data using a hash function.
Examples: SHA256, SHA1, MD5.
I Sign hash with private key, verify with public
key. Examples: RSA, ECDSA.
I Root CAs are self-signed and installed by the
OS vendor or local admin (Group Policy, etc.).
7
TLS handshake with RSA key exchange method
I Client Hello advertises supported parameters, Server Hello decides.
I Server picked RSA key exchange: TLS RSA WITH AES 128 CBC SHA.
+ Certificate (with RSA public key)
+ ServerHelloDone
8
TLS handshake with RSA key exchange method - ClientKeyExchange
I Client receives Server Hello, knows protocol version and cipher suite.
I Client generates a new random 48-byte premaster secret, encrypts it using the
public key from the Certificate and sends the result to the server in a
ClientKeyExchange message.
I Using the private RSA key, server (or anyone else!) decrypts the premaster secret.
9
TLS handshake with RSA key exchange method - Finishing up
I Both sides calculate the 48-byte master
secret based on the Client Random,
Server Random and the premaster secret.
I Both sides derive symmetric keys from this
master secret, send the ChangeCipherSpec
message to start record protection.
I Finally they both finish the Handshake
protocol by sending a Finished Handshake
message over the encrypted record layer.
I Now the actual encrypted Application
Data can be sent and received.
10
Handshake overview
Client Server
ClientHello -------->
ServerHello
Certificate*
ServerKeyExchange*
<-------- ServerHelloDone
ClientKeyExchange
[ChangeCipherSpec]
Finished -------->
[ChangeCipherSpec]
<-------- Finished
Application Data <-------> Application Data
Simplified TLS handshake (adapted from RFC 5246 (TLS 1.2)) 11
Plaintext please
I Server administrators can check application logs.
I Web browsers provide developer tools.
I What if the information is not logged?
I What if you want to know what this third-party Android app is doing?
I What if the application under investigation is poorly documented?
I What if you want to debug your new HTTP/2 feature?
I Solution: packet capture plus SSL/TLS secrets!
12
Decryption using private RSA server key
Configure Wireshark with RSA private key file1 :
I IP address is unused and ignored. Port
+ Protocol can be empty. These three
fields will be removed in future.
I Specify PEM-encoded key file or
PKCS#12 Key File + Password.
1 13
See [Link]
Limitations of RSA private key
I Clients usually do not have access to the RSA key, only server operators can use it.
I In case of mutual authentication (client certificates), the private key is only used
for signing. The client private RSA key cannot decrypt.
I Encrypted premaster secret is not sent with resumed sessions.
Client Server
ClientHello -------->
ServerHello
[ChangeCipherSpec]
<-------- Finished
[ChangeCipherSpec]
Finished -------->
Application Data <-------> Application Data
Message flow for an abbreviated handshake (RFC 5246, Figure 2) 14
Ephemeral (Elliptic Curve) Diffie-Hellman (ECDHE)
I Decryption using RSA private key not possible with cipher suites like
TLS ECDHE ECDSA WITH AES 128 GCM SHA256 and
TLS ECDHE RSA WITH AES 128 GCM SHA256.
I Although it has RSA in its name, it is not used for encryption, but signing.
I Instead it uses Diffie-Hellman to establish a shared secret (the premaster secret)
based on ephemeral secrets (different secrets for every session).
I Server chooses a group/curve, generates private value and its related public value
and sends it to the client. Client uses same group/curve and also generates a pair.
I Computationally hard to find the private value given the public one.
15
TLS secrets summary
I Any of these can be used for decryption with passive captures:
I premaster secret: RSA-encrypted or output from DH key exchange
I Master secret: derived from premaster secret and handshake context. Also used for
session resumption.
I Symmetric encryption key for record encryption.
I RSA private key file (for RSA key exchange, covered before).
I So how to use the master secrets?
16
SSL key log file
I Text file with master secrets2 .
I Works for any cipher, including RSA and DHE.
I Clients can use this too!
I Set environment variable SSLKEYLOGFILE before starting Firefox or Chrome. The
variable is only read during startup, so restart if necessary.
I Format: CLIENT RANDOM <Client Hello Random> <master secret> .
# SSL/TLS secrets log file, generated by NSS
CLIENT_RANDOM 5f4dad779789bc5142cacf54f5dafba0a06235640796f40048ce4d0d1df63ad8 a4d69a3fa4222d6b6f2492e66dca2b1fc4e2bc143df849ad45eff9f
CLIENT_RANDOM c2407d5ba931798e3a35f775725fb3e5aefcb5804bb50271fe3bd5fb19c90061 e419759e7b44f766df6defe6b656eda3d430754044773b6fc0a91eb
CLIENT_RANDOM abec6cf83ea1dcb135b21fd94bc0120dd6a37dca0fcd96efd8989d05c51cc3ab 5b4d525dfe3168132d388881033633c2aba99346c25ae8163f2191f
CLIENT_RANDOM dffe2c85a7d6f3c3ec34ba52ea710f0f1649e58afa02f9824d983ea74f07900e fdb58d49482f876f200ce680b9d6987434e3aca54d203fc57cc5888
CLIENT_RANDOM fbf40ada961093cd917fba97bfffe7c4b0bbf57a0cf90626dee417d3d12b3755 6b4e313d6be9316c42f47ddd3ceeef9743825bd3c3bb25ec9ac73c9
CLIENT_RANDOM 2b8184f7642df4bb5979ad9a623690b08f392deb94fdb64b00d7dc78b711638b dfdbe9f4d6949eea02489eb39b2c8d7770c12928becaf0ac1e34edf
CLIENT_RANDOM 7e4340c76c720d39c98e761697be0f32e1c79c6c04ade05a3f29325ac9cae612 1dfe402b85560048ae278b78febe83ee1640785b969c328d94a785a
2 17
File format at [Link]
Using SSL key log file in Wireshark
I Configure file in Wireshark preferences: Edit → Preferences; Protocols → SSL;
(Pre-)Master Secret log filename.
I Key log file is also read during a live capture. And if the file is removed and a new
file is written, the new key log file is automatically read.
I Caveat: key log is read while processing ChangeCipherSpec. If key is written too
late, trigger a redissection (e.g. change a preference or (Un)ignore a packet).
18
Application and library support
I Any application built using NSS and GnuTLS enable key logging via the
SSLKEYLOGFILE environment variable.
I Applications using OpenSSL 1.1.1 or BoringSSL d28f59c27bac (2015-11-19) can
be configured to dump keys:
v o i d S S L C T X s e t k e y l o g c a l l b a c k ( SSL CTX ∗ c t x ,
v o i d ( ∗ cb ) ( c o n s t SSL ∗ s s l , c o n s t c h a r ∗ l i n e ) ) ;
I cURL supports many TLS backends, including NSS, GnuTLS and OpenSSL. Key
logging with OpenSSL/BoringSSL is possible with curl 7.56.03
3 19
Requires a build time option, see [Link]
Key log with OpenSSL 1.1.0 and before
I Why: many applications (including Python) use OpenSSL.
I Problem: older OpenSSL versions have no key log callback.
I Solution: intercept library calls using a debugger or an interposing library
(LD PRELOAD) and dump keys4 .
I Example with OpenSSL 1.1.0f using an intercepting library5 :
$ export SSLKEYLOGFILE=[Link] LD_PRELOAD=./[Link]
$ curl [Link]
...
$ cat [Link]
CLIENT_RANDOM 12E0F5085A89004291A679ABE8EE1508193878AB9E909745CA032212FCA24B89 148AF5875F83
4
[Link]
5 20
[Link]
Unsupported applications for SSLKEYLOGFILE
I Windows native TLS library is Secure Channel (SChannel). Pending feature
request for Microsoft Edge browser: 6
I Extracting secrets from SChannel is not impossible (but neither easy) though7
I Apple macOS applications use SecureTransport, also not supported.
6
[Link]
16310230-ssl-key-logging-aka-sslkeylogfile
7
[Link]
21
[Link]
Alternative ways to get the secret
I Force RSA key exchange (disable forward-secret cipher suites).
I Setup a fake CA and force traffic through a proxy like mitmproxy8 , OWASP Zap,
Fiddler or Burp Suite.
I All of these methods can be detected by the client. Certificate pinning can also
defeat the custom CA method.
I The proxy interception method may also weaken security9 .
I If you are really serious about a passive, nearly undetectable attack from a
hypervisor, see the TeLeScope experiment10 .
8
[Link]
9
Durumeric et. al., The Security Impact of HTTPS Interception,
[Link]
10
[Link]
22
telescope-peering-into-the-depths-of-tls-traffic-in-real-time/
Feature: Follow SSL Stream
I Display the contents of the
decrypted application data.
I Right-click in the packet list or
details view, Follow → SSL Stream.
I Great for text-based protocols like
SMTP. For binary data, try the Hex
Dump option.
I Click on data to jump to related
packet (in packet list). Note that
display filter can hide packet, clear
the filter to avoid that.
23
Feature: Export HTTP Objects
I After decryption is enabled, HTTP
payloads within TLS (HTTPS) can
be exported.
I File → Export Objects → HTTP. . .
I Click on an item to select it in the
packet list.
I Note: does not cover HTTP2 or
QUIC (yet?) as of Wireshark 2.4.
24
Feature: Export SSL Session Keys
I Suppose you have a capture which is decrypted using a RSA private key file. How
to allow others to decrypt data without handing over your RSA private key file?
I File → Export SSL Session Keys. . .
I Generates a key log file which can be used instead of the private RSA key file.
I Note: currently contains all keys. Remove lines which are not needed (match by
the second field, the Random field from Client Hello).
25
Feature: Display Filters
I Display filters can be used for filtering, columns and coloring rules.
I Discover by selecting a field in packet list, look in status bar.
I Recognize TCP/TLS stream in packet list: Right-click TCP Stream Index
([Link]) field in packet details, Apply as Column.
I Right-click field in packet details, Apply/Prepare as Filter.
I SNI in Client Hello: [Link] server name
I Change in Wireshark 2.4: [Link] selects full Client or Server
Random instead of the just the Random Bytes field. Reason: real time is often no
longer included, full bytes field is useful for matching with key log file.
26
Feature: Decode As
I Force dissector for custom ports. Decode as SSL (TCP) or DTLS (UDP).
I Select application data protocol within SSL/TLS layer (since Wireshark 2.4).
I Example: HTTPS on non-standard TCP server port 4433.
I Right-click TCP layer, Decode As. Change current protocol for TCP Port to SSL.
I Press OK to apply just for now or Save to persist this port-to-protocol mapping.
I Right-click SSL layer, Decode As. Change current protocol for SSL Port to HTTP.
I For STARTTLS protocols, select SMTP/IMAP/. . . instead of SSL for TCP Port.
I Tip: there are many protocols, just select the field, then use arrow keys or type
the protocol name (typing H gives HTTP).
27
Feature: Tshark
I Tshark: command-line tool, useful to extract information as text, especially when
the query is repeated multiple times.
I Find all cipher suites as selected by the server: tshark -r [Link]
-Tfields -e [Link] -Y [Link]==2
I List all protocol fields: tshark -G fields
I Configure keylogfile:
tshark -[Link] file:[Link] -r [Link]
I Configure RSA keyfile (fields correspond to the RSA keys dialog):
wireshark -ouat:ssl keys:’"","","","keys/[Link]",""’
I Tshark manual: [Link]
28
Future: TLS 1.3
I Replaces all previous cipher suites with new one. Dropped all old cipher suites (no
more CBC, RC4, NULL, export ciphers).
I RSA key exchange is gone, all ciphers are forward secret.
I Encrypted early (0-RTT) data.
I Encrypted server extensions (like ALPN)
I Encrypted server certificate.
I Multiple derived secrets for resumption, handshake encryption, application data
encryption. (Safer resumption!)
I Decryption and dissection is supported by Wireshark 2.4 (drafts 18-21 as of
Wireshark 2.4.2).
29
Conclusions
I RSA private keys cannot be used for decryption in all cases.
I The key log method (SSLKEYLOGFILE) can also be used by clients and works with
all cipher suites.
I TLS 1.3 debugging is even more difficult without decryption.
I Use latest Wireshark version, especially if you are doing any TLS 1.3 work.
30
