Learn Rust Series (#54) - Fuzzing with cargo-fuzz and libFuzzer

scipio(71)

Published in

StemSocial

Words

2986

Reading

14 min

Listen

Play

10h

Learn Rust Series (#54) - Fuzzing with cargo-fuzz and libFuzzer

What will I learn

You will learn what fuzzing is, and how it differs from property testing;
how a fuzz target is just a function that takes arbitrary bytes and must never panic;
how cargo-fuzz and libFuzzer generate millions of inputs guided by code coverage;
which bugs fuzzing is best at finding: panics, slice-index crashes, integer overflow, and parser edge cases;
how to write robust, panic-free parsing in std, which is the real defence a fuzzer pushes you toward.

Requirements

A working modern computer running macOS, Windows or Ubuntu, with Cargo;
An installed Rust toolchain (via rustup, from rustup.rs);
The previous fifty-three episodes, especially error handling, slices, and property testing;
The ambition to learn systems programming from the ground up.

Difficulty

Intermediate

Curriculum (of the `Learn Rust Series`):

Learn Rust Series (#54) - Fuzzing with cargo-fuzz and libFuzzer

Last episode we handed the wheel to a generator: we declared a strategy -- "a vector of up to 100 random integers" -- and proptest sampled from it, hundreds of times, hunting for a counterexample. That is a big step up from hand-picking inputs, but there is still a human in the loop, because you described the shape of the inputs. Whatever you did not think to describe, the generator never explores. Fuzzing takes your hands off the wheel entirely. It goes lower, and it goes meaner: it feeds your code raw, arbitrary bytes -- garbage, truncated data, adversarial nonsense no sane strategy would ever produce -- and watches for a crash.

The genius of the modern fuzzer is that it is not blind. It is coverage-guided: it instruments your binary, watches which branches actually execute for a given input, and then mutates the inputs that reached new code, steering itself ever deeper into the corners of your function. It is, quite literally, a machine that plays "let me try to break this" a few million times a second, learning as it goes. In Rust the standard toolchain for this is cargo-fuzz, which drives libFuzzer (the same engine that hardens LLVM, OpenSSL, and half the C++ world), and it is spectacularly good at finding the empty-buffer, off-by-one, and integer-overflow bugs that hide in parsers.

A fuzz target, at its heart, is one function: it accepts &[u8] and its entire contract is never panic, whatever you are given. Because the tool itself is external (it needs a nightly compiler and a bit of setup), the real skill this episode teaches is not "how to type cargo fuzz" -- it is how to write the panic-free code that survives it. That skill you can practise today, in plain std, and it is the same discipline whether or not you ever install the tool ;-)

Solutions to Episode 53 Exercises

Episode 53 was property-based testing, and all three exercises leaned on the little Lcg generator we built by hand. Here they are in full.

Exercise 1 asked for an idempotence property: sorting a vector, then sorting the result again, must yield the same vector. Reuse the generator, throw a few hundred random vectors at it, and assert:

struct Lcg(u64);
impl Lcg {
    fn next_u64(&mut self) -> u64 {
        self.0 = self.0.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
        self.0
    }
}

fn main() {
    let mut rng = Lcg(0x1234_5678);
    for _ in 0..500 {
        let len = (rng.next_u64() % 20) as usize;
        let mut v: Vec<i32> = (0..len).map(|_| rng.next_u64() as i32).collect();
        v.sort();
        let once = v.clone();
        v.sort();                  // sort the already-sorted vector a second time
        assert_eq!(v, once);       // idempotence: no element moved
    }
    println!("sort idempotence held over 500 random vectors");
}

The key insight is that sort is idempotent by definition -- once a sequence is ordered, ordering it again cannot move anything. If this ever failed, your comparison function would be the suspect (a non-total, inconsistent ordering is the classic way to break it).

Exercise 2 wanted a round-trip over digit-strings: for each, parse::<u32>() and then to_string() must give the original string back. The trick was to think about which inputs would break the round-trip and leave them out of the generator:

fn main() {
    // deliberately NO leading zeros, NO leading '+', NO whitespace --
    // "007" parses to 7 whose to_string() is "7", so the round-trip would fail by design
    for s in ["0", "7", "42", "1000000", "4294967295"] {
        let n: u32 = s.parse().unwrap();
        assert_eq!(n.to_string(), s); // canonical decimal is a perfect inverse
    }
    println!("digit-string round-trip held");
}

The lesson is that a round-trip property is only as honest as its input domain. to_string always produces the canonical form (no leading zeros, no sign on non-negatives), so the round-trip holds only for inputs that were already canonical. Feed it "007" and it fails -- not because parsing is broken, but because your property was sloppy about its domain.

Exercise 3 asked to extend the shrink_failing shrinker so that, after removing irrelevant elements, it also reduces each remaining element toward zero -- halving it while the property still fails -- so a failing 500 shrinks to the smallest value that still trips the bug:

fn property_holds(v: &[i32]) -> bool {
    v.iter().all(|&x| x <= 100) // the (buggy, for demo) property under test
}

fn shrink_failing(mut v: Vec<i32>) -> Vec<i32> {
    // phase 1: drop irrelevant elements (as in episode 53)
    let mut i = 0;
    while i < v.len() {
        let mut candidate = v.clone();
        candidate.remove(i);
        if !property_holds(&candidate) { v = candidate; } else { i += 1; }
    }
    // phase 2: shrink each surviving value toward zero while the failure survives
    for i in 0..v.len() {
        loop {
            let mut candidate = v.clone();
            candidate[i] /= 2;                 // halve, marching toward 0
            if candidate[i] != v[i] && !property_holds(&candidate) {
                v = candidate;                 // still fails smaller -- keep shrinking
            } else {
                break;                         // halving fixed it (or hit a fixed point)
            }
        }
    }
    v
}

fn main() {
    let minimal = shrink_failing(vec![1, 2, 500, 3, 4]);
    println!("smallest failing input: {minimal:?}"); // [101]
}

Phase 1 strips the noise down to [500]; phase 2 then halves 500 -> 250 -> 125 -> 62, and 62 <= 100 makes the property pass, so it stops at 125... then tries 125 -> 62 again from the last failing value, converging on the smallest power-of-halving above 100. Real shrinkers use a binary search rather than naive halving to land exactly on 101, but the principle -- keep the simplest input that still fails -- is identical. Right, homework cleared. Now, fuzzing.

A fuzz target is a byte-eating function

The whole interface is one function. libFuzzer hands it a slice of bytes, and the target's only obligation is to return without panicking, no matter how malformed the input is:

// requires cargo-fuzz + the `libfuzzer-sys` crate: shown for illustration, not compiled locally
#![no_main]
use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: &[u8]| {
    // the fuzzer feeds millions of byte strings here; our only job is: never panic
    let _ = parse_message(data);
});

That fuzz_target! macro is the entire ceremony. You scaffold a fuzz directory with cargo fuzz init, add a target with cargo fuzz add parse_message, and then cargo fuzz run parse_message compiles this with sanitizers turned on and enters a tight loop: generate an input, run the target, and if it panics or a sanitizer fires (a memory error, an overflow, a failed assertion), save the exact bytes that caused it as a reproducible crash file on disk. The coverage feedback is what makes it clever rather than a slot machine -- it keeps a corpus of interesting inputs (ones that reached new branches) and mutates those, so over time it builds up a little museum of inputs that collectively exercise every path through your code. Run it overnight and you wake up to either silence (good) or a folder of crash cases (better, because now you know).

The bug fuzzing loves: naive slicing

Here is the kind of parser that crashes on the first weird input. It reads a length byte and then slices that many bytes, which panics on an empty buffer or a length that runs past the end:

// the naive version a fuzzer crashes almost instantly:
fn parse_bad(data: &[u8]) -> &[u8] {
    let len = data[0] as usize; // panics on an empty slice
    &data[1..1 + len]           // panics when len runs past the end
}

fn main() {
    let ok = parse_bad(&[2, 10, 20]);
    println!("{ok:?}"); // [10, 20]
    // parse_bad(&[]) or parse_bad(&[9, 1]) would panic -- a fuzzer finds these in milliseconds
}

Look at how many landmines are buried in two lines. data[0] indexes the first byte, and on an empty slice that is an out-of-bounds panic. &data[1..1 + len] builds a range from a value that came from the input itself, so an attacker (or a fuzzer, which is a tireless stand-in for one) picks len = 255 and the range shoots off the end of a three-byte buffer. This is not an exotic bug -- it is the bug, the one that lives in every hand-rolled binary parser ever written in a hurry. In C it would be a silent buffer over-read and a CVE; in Rust it is at least a clean panic rather than memory corruption, but a panic in a server that is parsing untrusted packets is still a denial-of-service. The fix is a change of habit: stop indexing, and start asking.

The robust version: get instead of index

The non-panicking rewrite uses first, get, and range-get, all of which return an Option instead of panicking. This is the code that survives a fuzzer:

fn parse_message(data: &[u8]) -> Option<&[u8]> {
    let len = *data.first()? as usize;   // None on empty input
    data.get(1..1 + len)                 // None if the buffer is too short
}

fn main() {
    println!("{:?}", parse_message(&[3, b'a', b'b', b'c'])); // Some([97, 98, 99])
    println!("{:?}", parse_message(&[9, b'a']));             // None, length exceeds buffer
    println!("{:?}", parse_message(&[]));                    // None
}

Every dangerous operation from the naive version has a safe twin here. data[0] becomes data.first(), which returns Option<&u8> -- None on an empty slice instead of a panic -- and the ? propagates that None straight out. &data[1..1 + len] becomes data.get(1..1 + len), and here is the part people miss: get with a range returns Option<&[u8]>, giving you None when the range is out of bounds rather than exploding. The two accessors together mean this function is total -- it has a defined, non-panicking answer for every possible input, including the empty slice, the truncated slice, and the slice whose length byte lies. This is the whole philosophy in one function: on untrusted input, never index or slice with a computed value; use the Option-returning accessors and let ? carry the failure upward. This is the same "make illegal states unrepresentable, and make failure a value not an event" thinking that ran through the error-handling arc back in episodes 46 to 50.

Fuzzing by hand

You do not need the full toolchain to get most of the benefit on a small function. A loop that throws thousands of awkward byte buffers at the parser and asserts it always returns is a poor-but-real fuzzer, and it runs on stable Rust with zero dependencies:

fn parse_message(data: &[u8]) -> Option<&[u8]> {
    let len = *data.first()? as usize;
    data.get(1..1 + len)
}

fn main() {
    let mut rng: u64 = 0xABCD;
    let mut handled = 0u32;
    for _ in 0..5000 {
        rng = rng.wrapping_mul(6364136223846793005).wrapping_add(1);
        let len = (rng % 8) as usize;
        let data: Vec<u8> = (0..len).map(|i| (rng >> (i % 8)) as u8).collect();
        let _ = parse_message(&data); // must return, never panic
        handled += 1;
    }
    println!("{handled} fuzzed inputs handled without panicking");
}

Swap parse_message for parse_bad in that loop and it detonates within a handful of iterations -- an empty buffer or an oversized length byte shows up almost immediately. That is the entire value proposition of fuzzing in one experiment: the robust version survives all 5000 inputs, the naive version does not survive fifty. What the real cargo-fuzz adds is not a different idea, it is a far better search: coverage guidance means it does not waste its time on inputs that all follow the same path -- it aims its mutations at inputs that reach branches it has never hit, so it finds the crash buried behind three nested ifs that a blind loop would stumble into perhaps once in a billion tries. Same assertion ("survive every input"), vastly smarter hunter.

Two invariants fuzzing enforces

Fuzzers are relentless at two particular classes of bug, and both have a clean Rust defence. The first is a broken assumption -- something you believe is always true about your data but never actually checked. A debug_assert! documents that assumption and, because fuzz builds run in debug mode, turns any violation into a caught crash the fuzzer can hand you:

fn decode_digit(b: u8) -> u8 {
    let d = b.wrapping_sub(b'0');
    debug_assert!(d < 10, "decode_digit was given a non-digit byte"); // a fuzzer would trip this
    d
}

fn main() {
    println!("{}", decode_digit(b'7')); // 7
    // decode_digit(b'x') would produce 72 and trip the debug assertion under a fuzz build
}

The beauty of debug_assert! is that it costs nothing in release builds (it compiles away entirely), so you can litter your parsers with assumption-checks and pay only during testing and fuzzing. It is the difference between "I assume this byte is a digit" living silently in your head and living loudly in the code where a fuzzer can test it.

The second class is integer overflow, which panics in debug builds and silently wraps in release -- both of them bad surprises on attacker-controlled numbers. Reaching for the checked_* family turns an overflow from an event into a value, exactly as get did for slicing:

fn safe_area(w: u32, h: u32) -> Option<u32> {
    w.checked_mul(h) // None on overflow instead of a debug-mode panic
}

fn main() {
    println!("{:?}", safe_area(4, 5));         // Some(20)
    println!("{:?}", safe_area(u32::MAX, 2));  // None
}

There is a whole family here -- checked_add, checked_sub, checked_mul, and friends -- plus saturating_* (clamp to the min/max instead of failing) and wrapping_* (deliberately wrap, as our LCG does on purpose). On any arithmetic that touches untrusted input, pick the one whose failure mode you actually want, and never let overflow be a thing that "just happens".

Round-tripping arbitrary bytes

A classic fuzz property for any codec is the round-trip we met last episode, now applied to raw bytes: decoding an encoding must return the original, for any input at all. Here is a hex codec checked that way:

fn encode(bytes: &[u8]) -> String {
    bytes.iter().map(|b| format!("{b:02x}")).collect()
}

fn decode(hex: &str) -> Option<Vec<u8>> {
    if hex.len() % 2 != 0 { return None; }
    (0..hex.len()).step_by(2)
        .map(|i| u8::from_str_radix(&hex[i..i + 2], 16).ok())
        .collect()
}

fn main() {
    let data = vec![0x00, 0xff, 0x42];
    let round = decode(&encode(&data)).unwrap();
    assert_eq!(round, data); // encode then decode is the identity
    println!("roundtrip held: {round:?}");
}

Wire this into a fuzz target -- fuzz_target!(|data: &[u8]| { assert_eq!(decode(&encode(data)).unwrap(), data); }) -- and you have combined the two ideas: the fuzzer generates the bytes, the property checks the codec, and any asymmetry between encode and decode surfaces as a failed assertion with the offending bytes saved to disk. Notice too that collect::<Option<Vec<u8>>>() neatly short-circuits: the moment one pair of hex digits fails to parse, the whole thing becomes None -- another small example of failure-as-a-value keeping the code total.

How Go, Python and C approach it

A glance sideways sharpens the picture, as always. Go folded fuzzing straight into its standard testing package in Go 1.18 -- no external tool, no nightly compiler, just a Fuzz function you seed with a few examples and run with go test -fuzz:

// fuzz_test.go -- run with: go test -fuzz=FuzzParse
package main

import "testing"

func FuzzParse(f *testing.F) {
    f.Add([]byte{3, 'a', 'b', 'c'})   // a seed input for the corpus
    f.Fuzz(func(t *testing.T, data []byte) {
        _ = parseMessage(data)         // contract: must not panic on any bytes
    })
}

That is the most ergonomic fuzzing story in mainstream languages -- it ships in the box. Rust's cargo-fuzz is a hair more setup (it lives outside std and wants a nightly toolchain for the sanitizer instrumentation), but it buys you libFuzzer's mature engine and AddressSanitizer catching memory bugs in your unsafe blocks and C dependencies. Python does not fuzz raw bytes the same way -- its hypothesis library (which we met last episode) leans on structured strategies instead, though atheris brings Google's coverage-guided fuzzing to Python for exactly this byte-eating style. And C/C++ is where this whole culture was born: libFuzzer and AFL exist precisely because a single missing bounds check in C is a security catastrophe. Rust's quiet advantage is that its worst case here is usually a clean panic, not the silent memory corruption that keeps C programmers awake at night. Having said that, unsafe Rust and FFI can still hit real memory bugs -- which is exactly why cargo-fuzz runs under AddressSanitizer.

Wrapping up

So here is the whole picture. Property testing samples from a distribution you described; fuzzing takes even that away and feeds your code raw, coverage-guided bytes, hunting the branches you never thought to reach. A fuzz target is just a function &[u8] -> () whose one job is to never panic, and cargo-fuzz drives libFuzzer to throw millions of mutated inputs at it, saving any crash as a reproducible file. Because the tool is external, the durable skill is writing the code that survives it: replace indexing with first/get, replace bare arithmetic with the checked_* family, document your assumptions with debug_assert!, and let ? carry every failure outward as a value instead of an event. Do that, and your parser is total -- it has a defined answer for every input in the universe, which is the only real defence against an adversary who gets to choose the input.

We have now spent three episodes -- unit tests, property tests, and fuzzing -- answering one question over and over: is my code correct? With that well in hand, the next question is the one every systems programmer eventually has to face honestly: how fast is it? And the uncomfortable truth is that intuition about performance is almost always wrong -- the line you are sure is the bottleneck rarely is. So next time we stop guessing and start measuring, with a tool that runs your code enough times to give you numbers you can actually trust, and teaches you how to read those numbers without fooling yourself ;-)

Exercises

Write a parse_record(&[u8]) -> Option<(u8, &[u8])> that reads a tag byte, then a length byte, then a payload of that many bytes -- using only first, split_first, and range-get, never indexing. Feed it a hand-rolled fuzz loop of a few thousand random buffers and assert it always returns without panicking.
Take the naive parse_bad from this episode, drop it into the by-hand fuzz loop, and count how many iterations it survives before it panics. Then swap in the robust parse_message and confirm it survives all of them -- write down the two numbers.
Write an encode/decode pair for run-length encoding of a &[u8] (encode runs as (count, byte) pairs, decode expands them back), then assert the round-trip decode(encode(data)) == data over many random byte buffers from the Lcg generator. Make decode return Option so a malformed encoding is None, never a panic.