How to decrypt WhatsApp end-to-end media files

At the center of the "Saudis hacked Bezos" story is a mysterious video file investigators couldn't decrypt, sent by Saudi Crown Prince MBS to Bezos via WhatsApp. In this blog post, I show how to decrypt it. Once decrypted, we'll either have a smoking gun proving the Saudi's guilt, or exoneration showing that nothing in the report implicated the Saudis. I show how everyone can replicate this on their own iPhones.

The steps are simple:

• backup the phone to your computer (macOS or Windows), using one of many freely available tools, such as Apple's own iTunes app
• extract the database containing WhatsApp messages from that backup, using one of many freely available tools, or just hunt for the specific file yourself
• grab the .enc file and decryption key from that database, using one of many freely available SQL tools
• decrypt the video, using a tool I just created on GitHub

End-to-end encrypted downloader

The FTI report says that within hours of receiving a suspicious video that Bezos's iPhone began behaving strangely. The report says:

...analysis revealed that the suspect video had been delivered via an encrypted downloader host on WhatsApp’s media server. Due to WhatsApp’s end-to-end encryption, the contents of the downloader cannot be practically determined. 

The phrase "encrypted downloader" is not a technical term but something the investigators invented. It sounds like a term we use in malware/viruses, where a first stage downloads later stages using encryption. But that's not what happened here.

Instead, the file in question is simply the video itself, encrypted, with a few extra bytes due to encryption overhead (10 bytes of checksum at the start, up to 15 bytes of padding at the end).

Now let's talk about "end-to-end encryption". This only means that those in middle can't decrypt the file, not even WhatsApp's servers. But those on the ends can -- and that's what we have here, one of the ends. Bezos can upgrade his old iPhone X to a new iPhone XS by backing up the old phone and restoring onto the new phone and still decrypt the video. That means the decryption key is somewhere in the backup.

Specifically, the decryption key is in the file named 7c7fba66680ef796b916b067077cc246adacf01d in the backup, in the table named ZWAMDIAITEM, as the first protobuf field in the field named ZMEDIAKEY. These details are explained below.

WhatsApp end-to-end encryption of video

Let's discuss how videos are transmitted using text messages.

We'll start with SMS, the old messaging system built into the phone system that predates modern apps. It can only send short text messages of a few hundred bytes at a time. These messages are too small to hold a complete video many megabytes in size. They are sent through the phone system itself, not via the Internet.

When you send a video via SMS what happens is that the video is uploaded to the phone company's servers via HTTP. Then, a text message is sent with a URL link to the video. When the recipient gets the message, their phone downloads the video from the URL. The text messages going through the phone system just contain the URL, an Internet connection is used to transfer the video.

This happens transparently to the user. The user just sees the video and not the URL. They'll only notice a difference when using ancient 2G mobile phones that can get the SMS messages but which can't actually connect to the Internet.

A similar thing happens with WhatsApp, only with encryption added.

The sender first encrypts the video, with a randomly generated key, before uploading via HTTP to WhatsApp's servers. This means that WhatsApp can't decrypt the files on their servers.

The sender then sends a message containing the URL and the decryption key to the recipient. This message is encrypted end-to-end, so again, WhatsApp itself cannot decrypt the contents of the message.

The recipient downloads the video from WhatsApp's server, then decrypts it with the encryption key.

Here's an example. A friend sent me a video via WhatsApp:

All the messages are sent using end-to-end encryption for this session. As described above, the video itself is not sent as a message, only the URL and a key. These are:

mediaurl = https://mmg-fna.whatsapp.net/d/f/AsnGB7gNh6Yw52MScbJyTRMo3NCmzMpesUIYyFmEZ0lR.enc

mediakey = TKgNZsaEAvtTzNEgfDqd5UAdmnBNUcJtN7mxMKunAPw=

These are the real values from the above exchange. You can click on the URL and download the encrypted file to your own computer. The file is 22,161,850 bytes (22-megabytes) in size. You can then decrypt it using the above key, using the code shown below. I can't stress this enough: you can replicate everything I'm doing in this blogpost, to do the things the original forensics investigators hired by Bezos could not.

iPhone backups and file extraction

The forensics report in the Bezos story mentions lots of fancy, expensive tools available only to law enforcement, like Celebrite. However, none these appear necessary to produce their results. It appears you can get the same same results at home using freely available tools.

There are two ways of grabbing all the files from an iPhone. One way is just to do a standard backup of the phone, to iCloud or to a desktop/laptop computer. A better way is to jailbreak the phone and get a complete image of the internal drive. You can do this on an iPhone X (like Bezos's phone) using the 'checkm8' jailbreak. It's a little complicated, but well within the abilities of techies. A backup gets only the essential files needed to restoring the phone, but a jailbreak gets everything.

In this case, it appears the investigators only got a backup of the phone. For the purposes of decrypting WhatsApp files, it's enough. As mentioned above, the backup needs these keys in order to properly restore a phone.

You can do this using Apple's own iTunes program on Windows or macOS. This copies everything off the iPhone onto your computer. The intended purpose is so that if you break your phone, lose it, or upgrade to the latest model, you can easily restore from this backup. However, we are going to use this backup for forensics instead (we have no intention of restoring a phone from this backup).

So now that you've copied all the files to your computer, where are they, what are they, and what can you do with them?

Here's the location of the files. There's two different locations for Windows, depending upon whether you installed iTunes from Apple or Microsoft.

• macOS: /Users/username/Library/Application Support/MobileSync/Backup
• Windows: /Users/username/AppData/Roaming/Apple Computer/MobileSync/Backup
• Windows: /Users/username/Apple/MobileSync/Backup

The backup for a phone is stored using the unique ID of the phone, the UDID:

Inside the backup directory, Apple doesn't use the original filenames on the phone. Instead, it stores them using the SHA1 hash of the original filename. The backup directory has 256 subdirectories named 00, 01, 02, .... ff corresponding to the first byte of the hash, each directory containing the corresponding files.

The file we are after is WhatsApp's ChatStorage.sqlite file, whose full pathname on the iPhone hashes to "7c7fba66680ef796b916b067077cc246adacf01d".

On macOS, the Backup directory is protected. You have to go into the Security and Privacy settings to give the Terminal app "Full Disk Access" permissions. Then, copy this file to some other directory (like ~) where other apps can get at it.

Note that in the screenshot above, I also gave "iPhone Backup Extractor" permissions. This program provides a GUI that gives files their original names (like "ChatStorage.sqlite") instead of hashes 7c7fba666... It also has a bunch of built-in logic for extracting things like photos and text messages.

The point of this section is to show that getting these files is simply a matter of copying off your phone and knowing which file to look for.

Working with WhatsApp chat log

In the previous section, I describe how to backup the iPhone, and then retrieve the file ChatStorage.sqlite from that backup. This file contains all your chat messages sent and received on your iPhone. In this section, I describe how to read that file.

This file is an SQL database in standard "sqlite" format. This is a popular open-source projects for embedding SQL databases within apps and it's used everywhere. This means that you can use hundreds of GUIs, command-line tools, and programming languages to read this file.

I use "sqlitebrowser", which runs as a GUI on Windows, macOS, and Linux. Below is a screenshot. As you can see, the filename is the file we copied in the step above, the hash of the original name. I then click on "Browse Data" and select the table ZWAMEDIAITEM. I see a list of those URLs in the column ZMEDIAURL, and the corresponding decryption keys in the column ZMEDIAKEY.

The media keys are "blobs" -- "binary large objects". If I click on one of those blobs I see the following as the mediakey:

This binary data is in a format called protobuf. The byte 0x0a means the first field is a variable length string. The next byte 0x20 means the string is 32-bytes long. The next 32-bytes is our encryption key, which I've highlighted. The next field (0x12 0x20) is a hash of the file. There are two more fields at the end, but I don't understand what they are.

So in hex, our encryption key is:

4ca80d66c68402fb53ccd1207c3a9de5401d9a704d51c26d37b9b130aba700fc

Or if encoded in BASE64;

TKgNZsaEAvtTzNEgfDqd5UAdmnBNUcJtN7mxMKunAPw=

We now have the mediaurl and mediakey mentioned above. All we need to do is download the file and decrypt it.

How to decrypt a WhatsApp media file

Now we come to the meat of this blogpost: given a URL and a key, how do we decrypt it? The answer is "unsurprising crypto". It's one of most important principles of cryptography that whatever you do should be something boring as normal, as is the case here. If the crypto is surprising and interesting, it's probably wrong.

Thus, the only question is which of the many standard ways did WhatsApp choose?

Firstly, they chose AES-256, which is the most popular choice for such things these days. It's key is 256-bits, or 32-bytes. AES is a "block cipher", which means it encrypts a block at a time. The block size is 16-bytes. When the final block of data is less than 16-bytes, it needs to be padded out to the full length.

But that's not complete. In modern times we've come to realize that simple encryption like this is not enough. A good demonstration of this is the famous "ECB penguin" [1] [2] [3]. If two 16-byte blocks in the input have the same cleartext data, they'll have the same encrypted data. This is bad, as it allows much to be deduced/reverse-engineered from the encrypted contents even if those contents can't be decrypted.

Therefore, WhatsApp needs not only an encryption algorithm but also a mode to solve this problem. They chose CBC or "cipher block chaining", which as the name implies, chains all the blocks together. This is also a common solution.

CBC mode solves the ECB penguin problem of two blocks encrypting the same way, but it still has the problem of two files encrypting the same way, when the first part of the files are the same. Everything up to the first difference will encrypt the same, after which they will be completely different.

This is fixed by adding what's called an initialization vector or nonce to the start of the file, some random data that's different for each file. This guarantees that even if you encrypt the same file twice with the same key, the encrypted data will still be completely different, unrelated. The IV/nonce is stripped out when the file is decrypted.

Finally, there is the problem with the encrypted file may be corrupted in transit -- accidentally or maliciously. You need to check this with a hash or message authentication code (aka MAC). In the case of WhatsApp, this will be in the first 10 bytes of the encrypted data, which we'll have to strip out at the end. This MAC is generated by using a different key than the AES key. In other words, we need two keys: one to encrypt the file, and a second to verify that the contents haven't been changed.

This explains why there was a 14 byte difference between the encrypted video and unencrypted video. The encrypted data needed 10 bytes for a MAC at the start, and 4 bytes for padding at the end.

The code

Here is the code that implements all the above stuff:

https://github.com/robertdavidgraham/whats-enc/blob/master/backend/whats-enc.py

At the top of the file I've hard-coded the values for the mediaurl and mediakey to the ones I found above in my iPhone backup.

The mediakey is only 32-bytes, but we need more. We need 32-bytes for the AES-256 key, another 16-bytes for the initialization vector, and 32-bytes for the message authentication key.

This is common problem is solved by using a special pseudo-randomization function to expand a small amount of data into a larger amount of data, in this case from 32-bytes to 112-bytes. The standard WhatsApp chose is "HMAC Key Derivation Function". This is expressed in my code as the following, where I expand the key into the IV, cipherkey, and mackey.:

mediaKeyExpanded=HKDF(base64.b64decode(mediaK),112,salt)

iv=mediaKeyExpanded[:16]

cipherKey= mediaKeyExpanded[16:48]

macKey=mediaKeyExpanded[48:80]

Then, I download the file from the URL. I have to strip the first 10 bytes from the file, which is the message authentication code.

mediaData= urllib2.urlopen(mediaurl).read()

file= mediaData[:-10]

mac= mediaData[-10:]

Then using the cipherkey from the first step, I decrypt the file. I have to strip the padding at the end of the file.

decryptor = AES.new(cipherKey, AES.MODE_CBC, iv)

imgdata=AESUnpad(decryptor.decrypt(file))

To download and decrypt the video, simply run the program as such:

I'm not going to link to the video myself. If you want to know what it contains, you are going to have to run the program yourself.

Remember that this example is a video a friend sent to me, and not the original video sent by MBS to Bezos. But the same principle applies. Simply look in that file in the backup, extract the URL and mediakey, insert into this program, and you'll get that file decrypted.

Conclusion

The report from FTI doesn't find evidence. Instead, it finds the unknown. It can't decrypt the .enc file from WhatsApp. It therefore concludes that it must contains some sort of evil malware hidden on that that encryption -- encryption which they can't break.

But this is nonsense. They can easily decrypt the file, and prove conclusively whether it contains malware or exploits.

They are reluctant to do this because then their entire report would fall apart. Their conclusion is based upon Bezos's phone acting strange after receiving that video. If that video is decrypted and shown not to contain a hack of some sort, then the rest of the reasoning is invalid. Even if they find other evidence that Bezos's phone was hacked, there would no longer anything linking to the Saudis.