Instrumenting masscan for AFL network fuzzing

This blog post is about work in progress. You probably don't want to read it.

---

So I saw this tweet today:

Fuzzing masscan by @ErrataRob with AFL by @lcamtuf.
Forgot to remove the crash-handler so crashes logged as hangs. pic.twitter.com/Yh5ElNyvOm

— Dr David D. Davidson (@dailydavedavids) June 5, 2016

As it turns it, he's just fuzzing input files. This is good, he's apparently already found some bugs, but it's not a huge threat.

Instead, what really needs to be fuzzed is network input. This is chronic problem with AFL, which is designed for inserting files, not network traffic, into programs.

But making this work is actually pretty trivial. I just need to make a tiny change to masscan so that instead of opening a libpcap adapter, it instead opens a libpcap formatted file.

This change was trivial, successfully running it is tough. You have to configure the command-line so all IP addresses match up with the libpcap file content, which is a pain. I created a sample lipcap file and checked it into the project, along with a help document explaining it. Just git clone the project, run make, then run this command line to see it run for yourself:

bin/masscan --nobacktrace --adapter file:data/afl-http.pcap --source-ip 10.20.30.200 --source-port 6000 --source-mac 00-11-22-33-44-55 --router-mac c0-c1-c0-a0-9b-9d --seed 0 --banners -p80 74.125.196.147 --nostatus

If you run on the command-line, it appears to return immediately. I say "appears" because there's actually a 10 millisecond wait. That limits fuzzing speed to a 100 attempts per second, rather than thousands per second. That's a tougher change, so I'll have to get around to fixing that, but in the meanwhile, you can just run a bunch of AFLs in parallel to get around this.

But when I try to run AFL, it's not working at the moment. In instead get this error:

As you can see, the command that returns in 10ms is now hanging when run under AFL, which says that it doesn't return in 1000ms. Using the '-t' option to increase the timeout doesn't help. Running masscan in some other way, such as parsing configuration files, works just fine.

---

Update

So I changed to where I "join" threads cleanly, so that the entire thing can run cleanly without every having to stop and wait. However, this creates a second problem not AFL refused to run because it's crashing instead of hanging. AFL suggests that it might be an out-of-memory issue, and that I should increase memory. So I bumped up memory and now it's running.

This memory issue might be what the problem was all along. Masscan assumes big scanning and sets up some large data structures at the start, so it may exceed the 50-megabyte assumed by AFL.

So now I have it running, fuzzing HTTP server response input:

But this isn't really success. The pcap file is 1986 bytes long. However, AFL has "trimmed" the input file down to 0.20%, which is the first 4 bytes of the file. This is just testing the libpcap library at this moment, and the fact that it supports multiple file types determined by the "magic" string in that first 4 bytes. I need to figure out how to make it fuzz starting deeper in the file, not at byte 0.

---

Update

I got the LLVM version working on Raspberry Pi 3 Odroid C2 ARM-64, so naturally I need to spread the work across 4 cores.

The Odroid has slightly faster CPUs than the Raspberry Pi 3, but mostly importantly, it's got 64-bit Linux available to it, which the Raspberry Pi 3 apparently still doesn't.