Hacking Flash Memory

I've got three computers now (Asus Eee PC, OLPC, and Dell M1330) that have flash disks instead of moving disks. In addition, my desktop has numerous flash chips to store the BIOS, network card ROM, video ROM, disk controller ROM, and so forth. It's time we look at the security risks of flash.

What we call "flash memory" is a type of memory that retains its contents after your turn the power off, as opposed to normal computer memory that loses its contents. In most cases, it's slower than normal memory, so it's more like a hard-disk than it is like normal memory. (NOR flash is fast but low capacity and is often used for BIOS ROMs, whereas NAND flash is slow buy high capacity, and is used for things like USB flash drives).

The major feature we are concerned about from a security point of view is that you cannot infinitely rewrite it. With normally memory and normal hard-disks, you can rewrite the contents trillions of times without concern. With flash memory, after rewriting data a few hundred thousand times, the block goes bad. It's quite easy to intentionally write a program that would continuously overwrite a block of flash until it failed.

At my previous employer, we found vulnerabilities in Cisco routers that would potentially allow us to create a worm. As part of our threat modeling, we considered what would happen if such a worm were to intentionally destroy the flash in a Cisco router. These routers boot from flash, so all the worm had to do was continuously overwrite the boot sector, then the router would no longer be able to boot.

The flash in many (if not most) Cisco routers is soldered in. Therefore, destroying the flash would "brick" the device. In other routers, the flash is socketed and can be replaced by the user.

Think of a hypothetical worm that infected the backbone routers and destroyed their flash chips. In one scenario, all those routers have soldered flash. It would take weeks or months for Cisco to send replacement motherboards to all those customers. In another scenario, all those routers have socketed flash, and the customers woul.d have spare flash modules on hand. In this scenario, they just simply install a (patched) copy of IOS on the flash modules, plug them in, and be back up and running within hours.

It's not just backbone ISPs that should take this threat into account, but anybody where the network is critical to operation. Financial networks (like the New York Stock Exchange) and the power grid are two examples that come to mind.

We therefore suggest that a element of critical infrastructure protection is to (1) require devices to have socketable flash, (2) for owners to have replaceable flash modules on hand, and (3) a way for owners to get the system back up and running after replacing the flash module.

Besides boot flash, mass storage flash has its own particular quirks. Take, for example, the notebook computers I mention above. A standard operating system continuously writes to a hard disk in the background. Windows frequently writes to the registry, and is constantly swapping memory to the disk. Linux is constantly updating /var/log files, and is constantly updating the atime record (last time you accessed a file). Thus, if you tried to boot Windows or Linux from a normal flash chip, you would quickly destroy the chip.

There is an obvious solution to this problem called "wear leveling". You create an extra layer between the operating system and the flash drive. Every time you rewrite a block of data, you instead write a new physical block. Thus, a logical disk location is different than the true physical location. This means all the blocks on the flask drive are overwritten equally, rather than any particular block being overwritten many times.

There are two ways of doing wear leveling: in the hardware, or in the software. You can now buy replacement IDE/SATA hard drives that replace your disks that have this wear leveling built-in. These are often called "solid state drives" or "SSDs". A good example is my Dell M1330, or Apple's new MacBook Air.

Whereas Windows is stuck with NTFS/FAT file systems that require to you solve wear-leveling in hardware, Linux can boot from special file systems that do wear-leveling in software. One popular flash file system is JFFS2. My Eee PC ($300 laptop) uses an SSD with hardware leveling, therefore, it boots using ext3 file system, and I can boot it with Windows if I wanted. My OLPC ($200 laptop) uses a cheaper flash chip without wear leveling. Therefore, it boots Linux with JFFS2, and I cannot install Windows on it.

The problem with either JFFS2 or SSDs is that a hacker can still attack the system and destroy the underlying flash and brick the system.

Besides boot flash or bulk storage, a hacker could also destroy the flash on your video card, network card, disk controller, and so forth. In some cases, the flash is only for optional features (like "network boot"), so destroying the flash won't impact most users. In other cases, the flash is a required part of the system. In order to boot from a SATA controller, the flash ROM must install its drivers at boot time.

Most desktops have a switch on the motherboard that would prevent a virus from destroying the BIOS flash, but such switches do not exist on laptops. There are likewise no switches protecting the other flash chips on the system whose destruction will prevent the system from booting. Again, we would suggest that for critical systems (such as servers), companies examine how well the flash is protected from destruction, and whether they can easily replace it in the field.

While destroying flash is easy, subverting is more fun. Errata Security conducted an experiment where we wrote a bit of malware under Windows that would subvert the flash of a network card. This flash would then subvert the boot process to in turn subvert a driver loaded from the hard disk, thus leaving a memory resident piece of code that could not be found on the hard disk. Re-installing the operating system did not remove the malware. We could also move the card to a new machine and infect that machine.

Such "flash rootkits" would therefore allow an attacker to take permanent control of a machine. Since the problem isn't addressed by security products, it's likely that the hackers can gain control without being detected, and if detected, would force the victim to replace the hardware completely.

There are some complications to subverting flash. Flash used for BIOSs are meant to be easily read and hard to write. Writing to flash often requires weird techniques. A typical example is that to switch the flash to write mode, you have to read from specific locations in the flash in a specific order to tell the chip you want to write, then do strange things to write data. Sometimes if you do all the correct things, the write operations will actually fail, so you have to try again until you get it right.

Luckily (for hackers), most systems do a raw pass through to the flash chips. A vendor of network cards, for example, wants to be able to choose different flash vendors, depending upon whoever has the cheapest chips at the moment. That means hackers don't need to worry about the quirks of a hundred different network chips - just the quirks of a few flash vendors. There is an open source project for flashing BIOSes that already contains the necessary code that a hacker could use to destroy/subvert almost any flash chip in any sort of addon card.

Most people don't have to worry about these threats, of course. Few hackers want to destroy the average person's systems. However, we would like to reiterate our belief that those responsible for protecting critical infrastructure take flash threats seriously. We believe they need to be able to respond a widespread infection that destroys the boot flash on critical devices.