Learn Zig Series):At the very end of episode 100 I promised we'd point our fan-out machinery at a new question. The port scanner asked "which doors are open"; today we ask something meaner -- "how many requests per second can this thing take before it starts to buckle, and how much do the slow requests hurt?" That's a load tester, and if you've ever run wrk, hey, ab, or oha against a service before shipping it, you've held one. We're going to build our own, from a raw socket up, because it turns out a load tester is a tiny program hiding two genuinely subtle ideas -- honest timing and honest statistics -- and both are worth seeing with your own eyes.
Before anyone reaches for the pitchfork: point this ONLY at a server you own, or at a throwaway one you spin up locally. Hammering someone else's endpoint with thousands of requests is, at best, rude and, at worst, a denial-of-service attack with your name on the packets. Throughout this two-parter I test against a local HTTP server -- the very one we built back in episodes 51 through 54 -- running on 127.0.0.1. That's the whole point: measure your own thing under pressure so you learn where it breaks in a lab, not in production. Having said that, let's build the engine. Part 1 is the request, the timing, the worker pool, and the statistics. We get a real, working tool by the end of this post.
Here's the mental model, and getting it right up front saves you from building a tool that lies. A load tester does three things in a loop: it fires a request, it times how long that request took end to end, and it records whether the server answered correctly. Do that a few hundred thousand times across several workers, and the pile of timings is your answer. Two numbers fall out of the pile: throughput (requests per second -- how much work got done) and latency (how long each individual request took -- how much a user waited). They are not the same thing, and confusing them is the classic rookie mistake. A server can have glorious throughput and horrible tail latency at the same time: it finishes tons of work overall, but every fiftieth user waits two seconds because a queue backed up.
Which is exactly why the average latency is close to useless and we're going to compute percentiles in stead. If ten thousand requests average 5ms but the slowest 1% take 900ms, the average whispers "all is well" while your worst users are furious. The p99 -- the latency that 99% of requests came in under -- tells the truth the average hides. So our tool's job, boiled down: generate a controlled flood, timestamp every request honestly, and report p50/p95/p99 in stead of a comforting-but-lying mean. Let me name the pieces first (episode 6), because a clear type for "one request's outcome" makes every later function honest about what it produces.
const std = @import("std");
const posix = std.posix;
/// Everything we learn from firing exactly one request. This is the atom the
/// whole tool is built from -- workers produce a pile of these, and the stats
/// pass consumes it. Keeping timing + outcome together (episode 6) means one
/// record is self-describing: we never have to correlate two parallel arrays.
pub const Sample = struct {
latency_ns: u64, // wall-clock time from "start sending" to "done reading"
status: u16, // the HTTP status code we parsed, or 0 if we never got one
ok: bool, // did the request complete AND return a 2xx/3xx?
};
That ok flag deserves a word. A request can "succeed" at the socket level -- bytes went out, bytes came back -- while the server was actually screaming 500 Internal Server Error because it was overwhelmed. Under load, errors are data, not noise: a service that answers fast because it's fast is healthy; a service that answers fast because it's instantly returning 503s is on fire. So we separate "did the transport work" from "did the server actually serve", and only count the second as a real success.
Now the atom: fire a single HTTP/1.1 GET and time it. We're building the request by hand over a raw TCP socket, the same connect-write-read dance from episode 21, speaking the plaintext HTTP/1.1 we dissected in episode 84. No client library -- partly because this is a from-scratch series, but mostly because a load tester must measure the wire and only the wire. Every abstraction layer you add is latency you're accidentally benchmarking. We want the timer to wrap the socket calls and nothing else.
/// Fire one HTTP/1.1 GET to an already-resolved address and time it end to end.
/// `req` is the pre-built request bytes (we build them once, reuse forever).
/// Returns a Sample whether it succeeds or fails -- a failed request is still
/// a measurement, and dropping it would flatter the server we're testing.
fn oneRequest(addr: std.net.Address, req: []const u8, scratch: []u8) Sample {
var timer = std.time.Timer.start() catch unreachable; // monotonic clock (ep 70)
// Connect. If we can't even open a socket to the target under load, that
// itself is the finding -- record a failed sample, don't crash the run.
const stream = std.net.tcpConnectToAddress(addr) catch {
return .{ .latency_ns = timer.read(), .status = 0, .ok = false };
};
defer stream.close(); // one connection per request; hand the fd back always
// Write the whole request. writeAll loops until every byte is gone or it
// errors -- a short write on a busy socket is normal, not a failure.
stream.writeAll(req) catch {
return .{ .latency_ns = timer.read(), .status = 0, .ok = false };
};
// Read the response into our scratch buffer until the server closes (we sent
// "Connection: close", so EOF marks the end -- no chunked parsing needed).
var total: usize = 0;
while (total < scratch.len) {
const n = stream.read(scratch[total..]) catch break;
if (n == 0) break; // EOF: server closed, response is complete
total += n;
}
const elapsed = timer.read(); // STOP the clock before we do any parsing
const status = parseStatus(scratch[0..total]);
return .{
.latency_ns = elapsed,
.status = status,
.ok = status >= 200 and status < 400,
};
}
Two things I want to shout about. First, look at where timer.read() is called for the success path: immediately after the read loop, before we parse the status. Parsing is our CPU work, not the server's, and including it in the latency would be measuring our own tool in stead of the thing under test. The clock starts before connect and stops the instant the last byte is in -- that window, and nothing else, is what the server made a user wait. Second, notice that every failure path still returns a Sample with ok = false. This is the discipline from the previous section made concrete: a connection refused under load is not an exception to swallow, it's the single most important measurement in the whole run. Drop it and your averages get artificially better the more the server struggles, which is exactly backwards.
The status parser is deliberately tiny -- we only need the three digits after HTTP/1.1, not a full response parse:
/// Pull the status code out of a response's first line: "HTTP/1.1 200 OK".
/// Returns 0 if the response is malformed or truncated -- treated as a failure
/// upstream, which is the honest reading of "the server sent us garbage".
fn parseStatus(resp: []const u8) u16 {
const prefix = "HTTP/1.1 ";
if (resp.len < prefix.len + 3) return 0;
if (!std.mem.startsWith(u8, resp, prefix)) return 0;
const digits = resp[prefix.len .. prefix.len + 3];
return std.fmt.parseInt(u16, digits, 10) catch 0;
}
Building the request itself we do exactly once, up front, and every worker reuses the same immutable byte slice. Formatting a request string per-request would be allocation and CPU work polluting our timings -- the opposite of what a benchmark wants:
/// Build the raw HTTP/1.1 GET bytes once; all workers share this slice.
/// "Connection: close" makes the server close after replying, so our reader
/// sees a clean EOF (episode 84) instead of guessing at content length.
fn buildRequest(alloc: std.mem.Allocator, host: []const u8, path: []const u8) ![]u8 {
return std.fmt.allocPrint(alloc,
"GET {s} HTTP/1.1\r\n" ++
"Host: {s}\r\n" ++
"User-Agent: zig-loadtest/0.1\r\n" ++
"Connection: close\r\n\r\n",
.{ path, host },
);
}
A single timed request is a stopwatch. A load tester is that stopwatch run flat-out by a pool of workers for a fixed duration, and here is where episode 100 pays off directly -- the shape is the same fan-out we built for the port scanner, just aimed at time in stead of a port range. Back then each worker grabbed the next port from an atomic counter and stopped when the range ran out. Now each worker loops making requests and stops when a deadline passes. The stopping condition changed; the skeleton did not.
The coordination is even simpler than the scanner's, and that's the nice surprise. There's no shared counter to hand out work -- every worker just does the same request over and over. The only shared state is a single deadline (a nanosecond timestamp) and one atomic "keep going" flag we can flip to stop everyone at once. Each worker keeps its OWN list of Samples, so there is zero contention on the output while the test runs -- no mutex, no shared vector, nothing to serialize on. We only merge the per-worker piles together at the very end, when the threads have joined and there's nobody left to race.
/// Shared, read-only-ish state every worker sees. `running` is the single point
/// of coordination: main flips it false when the duration elapses, and each
/// worker checks it once per loop (episode 30's atomics -- no lock needed).
const Load = struct {
addr: std.net.Address, // pre-resolved target (resolve ONCE, like ep100)
request: []const u8, // pre-built bytes, shared immutably by all workers
running: *std.atomic.Value(bool),
deadline_ns: u64, // absolute monotonic time to stop, as a safety backstop
};
/// One worker: hammer the target until told to stop, recording every outcome
/// into ITS OWN list. Single-writer-per-list means no synchronisation at all
/// on the hot path -- the merge happens later, single-threaded.
fn worker(load: *Load, out: *std.ArrayList(Sample), scratch: []u8) void {
var clock = std.time.Timer.start() catch unreachable;
while (load.running.load(.monotonic)) {
// Second guard: even if the flag flip is a hair late, never run past
// the hard deadline. Two independent stop conditions, belt and braces.
if (clock.read() >= load.deadline_ns) break;
const sample = oneRequest(load.addr, load.request, scratch);
// append can only fail on OOM; if it does we simply stop recording
// rather than crash a run that's otherwise producing good data.
out.append(sample) catch break;
}
}
The .monotonic ordering on that atomic load is the same reasoning as episode 100: we are not using running to publish other memory to the worker, we just need each thread to eventually observe the flip from true to false. The weakest ordering that guarantees visibility is the correct (and cheapest) choice; paying for .seq_cst here would be superstition, not safety. Note also the belt-and-braces stop: the atomic flag is the primary "everybody halt", but each worker also checks a hard deadline, so a worker blocked in a slow read() when the flag flips can't accidentally fire one extra request into the next epoch. Little correctness details like that are the difference between a benchmark you trust and one you argue with.
Now the orchestration -- resolve the host once, spawn the pool, sleep for the test duration on the main thread, then flip the flag and join:
/// Run `worker_count` workers against `host:port``path`` for `duration_ms`.
/// Returns every Sample from every worker, merged. Caller frees the slice.
pub fn run(
alloc: std.mem.Allocator,
host: []const u8,
port: u16,
path: []const u8,
worker_count: usize,
duration_ms: u64,
) ![]Sample {
if (worker_count == 0) return error.NoWorkers;
// Resolve ONCE -- a per-request DNS lookup would add lookup latency to every
// sample and quietly ruin the measurement (same lesson as the port scanner).
const list = try std.net.getAddressList(alloc, host, port);
defer list.deinit();
if (list.addrs.len == 0) return error.NameResolutionFailed;
const request = try buildRequest(alloc, host, path);
defer alloc.free(request);
var running = std.atomic.Value(bool).init(true);
var load = Load{
.addr = list.addrs[0],
.request = request,
.running = &running,
.deadline_ns = duration_ms * std.time.ns_per_ms,
};
// One results list AND one read buffer per worker -- no sharing, no locks.
const lists = try alloc.alloc(std.ArrayList(Sample), worker_count);
defer alloc.free(lists);
const bufs = try alloc.alloc([]u8, worker_count);
defer alloc.free(bufs);
const threads = try alloc.alloc(std.Thread, worker_count);
defer alloc.free(threads);
for (lists, bufs, 0..) |*l, *b, i| {
l.* = std.ArrayList(Sample).init(alloc);
b.* = try alloc.alloc(u8, 64 * 1024); // 64 KiB response scratch each
threads[i] = try std.Thread.spawn(.{}, worker, .{ &load, l, b.* });
}
// The main thread is the referee: sleep the test out, then call time.
std.time.sleep(duration_ms * std.time.ns_per_ms);
running.store(false, .monotonic);
for (threads) |t| t.join();
// Everyone has stopped -- NOW it's safe to merge single-threaded.
var merged = std.ArrayList(Sample).init(alloc);
for (lists, bufs) |*l, b| {
try merged.appendSlice(l.items);
l.deinit();
alloc.free(b);
}
return merged.toOwnedSlice();
}
The structure is worth internalising because you'll reuse it constantly: spawn, sleep, signal, join, merge. The main thread does no work except keep time -- it's the referee holding the whistle. Each worker is fully independent right up until the join, at which point the concurrency is over and the merge is boring, safe, single-threaded slice-copying. Notice too that each worker gets its own 64 KiB scratch buffer allocated up front (episode 7); nobody allocates on the hot path, so the only thing our timings capture is the network, not the allocator warming up.
We now have a big slice of Samples. The port scanner's lesson applies verbatim here: don't test the network, test the logic that silently drifts -- and the fiddly, bug-prone logic in a load tester is the statistics, not the sockets. Computing a percentile has three off-by-one traps and an empty-input trap, and it's a pure function of a number slice, so it's exactly the piece we pin down with tests. The recipe: sort the latencies, then for percentile p, pick the element at rank ceil(p/100 * N) - 1, clamped into range.
/// Return the p-th percentile latency (0..100) from a slice of latencies.
/// MUTATES `values` by sorting it in place -- the caller owns that slice and
/// we're done reading it in original order, so an in-place sort is the cheap,
/// allocation-free choice. Empty input yields 0: no data, no latency.
pub fn percentile(values: []u64, p: f64) u64 {
if (values.len == 0) return 0;
std.mem.sort(u64, values, {}, std.sort.asc(u64));
if (p <= 0) return values[0];
if (p >= 100) return values[values.len - 1];
// rank = ceil(p/100 * N), then to a 0-based index. The @max(...,1) guards
// the p just above 0 case so we never land on index -1.
const rank = @ceil(p / 100.0 * @as(f64, @floatFromInt(values.len)));
const idx = @min(@as(usize, @intFromFloat(@max(rank, 1))) - 1, values.len - 1);
return values[idx];
}
Every line in there is guarding a specific mistake I have personally shipped at some point ;-) The empty check stops a divide-into-nothing crash. The p >= 100 short-circuit stops ceil from producing rank N and indexing one past the end. The @max(rank, 1) stops a tiny percentile from computing rank 0 and then underflowing to index -1 (which, usize being unsigned, would wrap to a gigantic number and panic). The final @min(..., len-1) is the last-line-of-defense clamp. Now the tests -- pure input, pure output, no sockets, deterministic every single run:
test "percentile picks the right ranks on a known set" {
var data = [_]u64{ 10, 20, 30, 40, 50, 60, 70, 80, 90, 100 };
// p50 over ten sorted values: ceil(0.5*10)=5, index 4 -> the value 50.
try std.testing.expectEqual(@as(u64, 50), percentile(&data, 50));
// p90: ceil(0.9*10)=9, index 8 -> 90. p100 is the max, p0 is the min.
try std.testing.expectEqual(@as(u64, 90), percentile(&data, 90));
try std.testing.expectEqual(@as(u64, 100), percentile(&data, 100));
try std.testing.expectEqual(@as(u64, 10), percentile(&data, 0));
}
test "percentile survives empty and single-element input" {
var empty = [_]u64{};
try std.testing.expectEqual(@as(u64, 0), percentile(&empty, 95));
var one = [_]u64{42};
try std.testing.expectEqual(@as(u64, 42), percentile(&one, 99));
}
That second test is the one I'd never delete. Empty input and single-element input are where percentile code goes to die, and both can absolutely happen in reality -- an empty run because the server was down and every connect refused, a one-sample run because the duration was too short. A load tester that panics when the server is completely dead is worse than useless: it crashes exactly when it has the most important thing to tell you. Two lines of test buy you a tool that reports "0 successful requests" calmly in stead.
With the engine done, main is almost anticlimactic -- pick the target and knobs, run the load, then reduce the pile into the summary a human actually reads:
pub fn main() !void {
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa.deinit();
const alloc = gpa.allocator();
// In a real CLI these come from std.process.args (episode 36's arg parsing).
const host = "127.0.0.1";
const samples = try run(alloc, host, 8080, "/", 32, 5000); // 32 workers, 5s
defer alloc.free(samples);
// Split successes from failures and collect just the latencies to rank.
var ok_count: usize = 0;
const lats = try alloc.alloc(u64, samples.len);
defer alloc.free(lats);
for (samples, 0..) |s, i| {
lats[i] = s.latency_ns;
if (s.ok) ok_count += 1;
}
const rps = @as(f64, @floatFromInt(samples.len)) / 5.0;
const stdout = std.io.getStdOut().writer();
try stdout.print("requests : {d} ({d} ok, {d} failed)\n", .{
samples.len, ok_count, samples.len - ok_count,
});
try stdout.print("throughput: {d:.0} req/s\n", .{rps});
try stdout.print("p50 : {d:.2} ms\n", .{ms(percentile(lats, 50))});
try stdout.print("p95 : {d:.2} ms\n", .{ms(percentile(lats, 95))});
try stdout.print("p99 : {d:.2} ms\n", .{ms(percentile(lats, 99))});
}
fn ms(ns: u64) f64 {
return @as(f64, @floatFromInt(ns)) / std.time.ns_per_ms;
}
Fire that at a local server (spin up the one from episodes 51-54 on port 8080) and you get precisely the terse verdict a load tester exists to produce:
$ zig build run
requests : 148213 (148213 ok, 0 failed)
throughput: 29642 req/s
p50 : 0.94 ms
p95 : 2.11 ms
p99 : 4.87 ms
Read that output like a doctor reads a chart. The p50 says the typical request was under a millisecond -- healthy. But the p99 being five times the median is the interesting bit: it means one request in a hundred waited noticeably longer, the fingerprint of a queue occasionally filling up. That gap between p50 and p99 is the single most useful thing this tool tells you, and it's the thing a plain average would have completely erased.
The cost model of the tester itself matters, because a benchmark that's bottlenecked on its own overhead measures itself in stead of the server -- a genuinely embarrassing failure mode. Our biggest honesty win is that the hot path allocates nothing: request bytes built once, one scratch buffer per worker reused every iteration, Samples appended into a list that amortises its growth. The timer wraps only the socket calls. So when the number says "29,642 req/s", that's the server's ceiling we're bumping into, not our formatter's.
The concurrency ceiling is the same one that bit the port scanner: file descriptors (episode 71). We open one fresh connection per request with Connection: close, which is the simplest correct model and fine for a Part 1, but it means every in-flight request holds a socket, and it churns through descriptors and TIME_WAIT states at a furious pace. Thirty-two workers stays comfortably under the usual 1024-descriptor ulimit, but crank the worker count into the thousands and you'll hit error.ProcessFdQuotaExceeded -- and, more subtly, you'll spend more time in the TCP handshake and teardown than in the request you meant to measure. Connection reuse (keep-alive) is the obvious next improvement, and not by accident, that's precisely the kind of refinement a Part 2 exists for.
One design call I'll defend: workers record failures as first-class Samples in stead of throwing them away. It's tempting to only keep the "good" requests so the latency numbers look clean, but that's measuring the server you wish you had. When a service falls over under load it does so by refusing connections and returning 5xx, and a load tester that hides those is malpractice. Our ok flag and the failed-request Samples mean the summary can say "40% of requests failed at 500 req/s" -- which is the entire reason you ran the test.
In C, the request is line-for-line the same syscalls -- getaddrinfo, socket, connect, write, read in a loop -- because Zig's std.net and posix are a thin skin over exactly those. The pool would be pthread_create plus an _Atomic flag, and it works. What C hands to your diligence is everything Zig checked: no defer stream.close() guaranteeing the descriptor comes back on every path (forget it under load and you leak sockets until the process dies mid-test), and no compiler forcing you to handle the connect-refused case in stead of dereferencing a bad fd. It's the same program with the guard rails unbolted.
In Rust, you'd likely reach for tokio and make each request an async task -- tokio::net::TcpStream with tokio::time::timeout, a JoinSet of thousands of futures in stead of a bounded thread pool. It's elegant and it scales past what OS threads comfortably do, and the borrow checker guarantees your per-task result collection has no data race. The cost is an async runtime and its dependency tree between you and the socket, plus the reality that you're now reasoning about a scheduler when you wanted to reason about read(). For a measurement tool, some people (me, on some days) prefer being able to see the exact syscall the timer wraps.
In Go, this is almost cheating -- http.Client with a timeout is the request, a sync.WaitGroup bounds the workers, a buffered channel gathers the Samples, and the GC erases every close and free we wrote by hand. If you want a working load tester before dinner, Go is the pragmatic pick, and half the real-world tools (hey, vegeta) are written in exactly it. What our from-scratch version buys is the same thing it always buys in this series: you can point at the precise byte where the clock starts, the precise read where it stops, and know with certainty what your number means. For learning, seeing the machinery is the value.
Step back and notice how little was actually new today. TCP connect-write-read is episode 21; the plaintext HTTP/1.1 we spoke is episode 84; the fixed-pool-of-workers fan-out is episode 100 with the stop condition swapped from "range exhausted" to "deadline passed"; the atomics coordinating the halt are episode 30; the monotonic timer is episode 70; the leak-checked tests are episode 26. We assembled a real, useful tool almost entirely out of parts we'd already sharpened -- which is rather the whole promise of getting a hundred episodes deep.
But this Part 1 tester is honest and naive. It opens a brand-new connection for every single request, which means a big chunk of every measured millisecond is TCP handshake, not the server's actual work -- we're partly benchmarking the kernel's connection setup. It reports latency only at the end, so a five-minute run tells you nothing until minute five. And it has no notion of a request rate you dial in; it just runs flat-out. Each of those is a real limitation with a real fix, and sanding them down -- reusing connections, streaming live stats, controlling the offered load in stead of just maxing it -- is exactly the work waiting for us next time. The engine runs today; next we make it tell the truth about fast servers too.
Bedankt en tot de volgende keer!
@scipio