Learn Zig Series (#168) - JPEG Decoder Basics

scipio(71)
Published in
StemSocial
Words
5052
Reading
23 min
Listen
Play
2h

Learn Zig Series (#168) - JPEG Decoder Basics

zig.png

What will I learn?

  • Why a JPEG is a completely different animal from a PNG -- lossy, frequency-domain compression in stead of lossless prediction -- and what that costs and buys;
  • The actual byte layout of a JFIF/JPEG file: the marker-and-segment framing, the special role of 0xFF, and how to walk it safely;
  • How the quantization tables (DQT) and Huffman tables (DHT) segments are laid out, and how to decode them into real Zig structs;
  • The heart of the format -- the 8x8 DCT block, the zig-zag order, dequantization, and a plain-integer inverse DCT that turns frequencies back into pixels;
  • Turning decoded YCbCr samples into RGB (the color-space work from episode 165 paying off again) and handing back an Rgba buffer for the Framebuffer from episode 156;
  • Testing a lossy codec sensibly, a few honest performance notes on where the time really goes, and how C, Rust and Go tackle the same decode.

Requirements

  • A working modern computer running macOS, Windows or Ubuntu;
  • An installed Zig 0.14+ distribution (download from ziglang.org) -- the code here is written against Zig 0.16;
  • The Rgba pixel type and Framebuffer from episode 156, the PNG decoder from episode 167, and the sRGB/color habits from episode 165;
  • Comfort with packed structs and bit work (episode 17), error unions (episode 4), and allocators (episode 7);
  • The ambition to learn Zig programming.

Difficulty

  • Advanced

Curriculum (of the Learn Zig Series):

Learn Zig Series (#168) - JPEG Decoder Basics

Last episode we read a PNG byte by byte and got real pixels in from disk. I closed by pointing at "another huge family of image files built on a completely different idea -- not lossless prediction but lossy frequency-domain compression". That family is JPEG, and today we pull it apart. Fair warning up front: a full baseline JPEG decoder is a bigger beast than a PNG decoder, so this is very much a "basics" episode -- I will walk the whole pipeline end to end, give you real, honest Zig for the parts that teach the most (the marker walk, the tables, the zig-zag, the inverse DCT, the color conversion), and be upfront about where a complete implementation would need more plumbing than a single Hive post can carry. By the end you will understand every stage a JPEG goes through, and have the pieces to grow this into a working decoder. Here we go!

But first, the three loose ends from the PNG episode.

Solutions to Episode 167 Exercises

Exercise 1 -- support grayscale. The PNG decoder scoped itself to 8-bit RGB and RGBA. Extending it to grayscale (color type 0) and grayscale+alpha (type 4) is mostly a matter of accepting those color types in the header and, on output, splashing the single gray sample across R, G and B. The unfilter machinery does not change one bit -- a filter operates on raw bytes and does not care what they mean:

const std = @import("std");
pub const Rgba = struct { r: u8, g: u8, b: u8, a: u8 = 255 };

// bpp is 1 (gray) or 2 (gray+alpha). `cur` is one already-unfiltered scanline.
fn emitGrayRow(cur: []const u8, width: usize, bpp: usize, out: []Rgba) void {
    var x: usize = 0;
    while (x < width) : (x += 1) {
        const gray = cur[x * bpp];
        out[x] = .{
            .r = gray,
            .g = gray,
            .b = gray,
            .a = if (bpp == 2) cur[x * bpp + 1] else 255,
        };
    }
}

test "grayscale expands into equal channels" {
    var out: [3]Rgba = undefined;
    emitGrayRow(&[_]u8{ 10, 128, 250 }, 3, 1, &out);
    for (out) |px| try std.testing.expect(px.r == px.g and px.g == px.b);
    try std.testing.expectEqual(@as(u8, 128), out[1].r);
}

The key insight is that grayscale is not a special decode path, only a special emit path -- the same predictors reconstruct the bytes and we reinterpret them at the very end. Keeping the format-specific decision (how many channels a sample has) in one small function is exactly the separation episode 167 leaned on.

Exercise 2 -- read the tEXt chunks. PNG stores metadata as ancillary tEXt chunks: a keyword, a single zero byte, then the Latin-1 value. Because they are ancillary, a file with none must still decode fine -- so we simply collect them as we walk, and never require them:

const std = @import("std");

// Given a tEXt chunk's data, split it at the first NUL into (keyword, value).
fn parseText(data: []const u8) ?struct { key: []const u8, value: []const u8 } {
    const nul = std.mem.indexOfScalar(u8, data, 0) orelse return null;
    return .{ .key = data[0..nul], .value = data[nul + 1 ..] };
}

test "tEXt splits keyword from value at the NUL" {
    const chunk = "Software\x00scipio-codec 1.0";
    const kv = parseText(chunk).?;
    try std.testing.expectEqualStrings("Software", kv.key);
    try std.testing.expectEqualStrings("scipio-codec 1.0", kv.value);
}

std.mem.indexOfScalar finds the separator without a hand-rolled loop, and returning an optional means a malformed tEXt with no NUL becomes a clean "skip it" in stead of a crash. In the full decoder you would push each parseText result into a std.StringHashMap (episode 22) hung off the returned image -- ancillary data, gathered when present, absent without complaint.

Exercise 3 -- verify the zlib Adler-32. We trusted std.compress.zlib to check the trailing checksum. Building Adler-32 yourself cements how zlib frames DEFLATE: two rolling sums modulo the largest prime below 65536 (which is 65521):

const std = @import("std");

fn adler32(bytes: []const u8) u32 {
    const MOD: u32 = 65521;
    var a: u32 = 1;
    var b: u32 = 0;
    for (bytes) |byte| {
        a = (a + byte) % MOD;
        b = (b + a) % MOD;
    }
    return (b << 16) | a;
}

test "adler32 of Wikipedia matches the known vector" {
    try std.testing.expectEqual(@as(u32, 0x11E60398), adler32("Wikipedia"));
}

a accumulates the running byte sum and b accumulates the running sum of a, so the checksum captures both what bytes appeared and in what order -- swap two bytes and b changes even though a does not. Taking the modulus every step keeps both sums from overflowing a u32; a production version defers the modulus and batches it for speed, but this form is the definition, plain to read. Right, the PNG chapter is properly closed. Now let us open something much stranger.

Why JPEG is a different animal

Here is the mental gear-shift you have to make. PNG is lossless: what you put in is exactly what you get out, byte-for-byte, because it compresses in the spatial domain -- predict a pixel from its neighbours, store the small error, let DEFLATE squash the runs. JPEG throws that overboard. It is lossy, and it works in the frequency domain. The insight it is built on is a fact about human eyes: we are far more sensitive to broad brightness changes than to fine, high-frequency detail, and more sensitive to brightness than to color. So JPEG transforms little blocks of the image into frequency components, then deliberately throws away the high-frequency ones the eye barely registers. That is where the compression comes from, and it is why a JPEG at quality 60 is a tenth the size of the PNG and still looks fine to you -- and why it is a terrible choice for a screenshot of sharp text, where the high frequencies are the whole point.

The baseline decode pipeline (the one we are studying) runs in reverse of the encoder, five stages deep:

  • Entropy decode: the compressed bitstream is Huffman-coded; decode it back into quantized DCT coefficients, 64 per 8x8 block.
  • Dequantize: multiply each coefficient by its quantization-table entry to undo the encoder's divide-and-round (this is the lossy step, run backwards).
  • Inverse DCT: turn each 8x8 block of frequency coefficients back into 8x8 spatial samples.
  • Upsample: chroma (color) is usually stored at half resolution; scale it back up to match luma (brightness).
  • Color convert: JPEG works in YCbCr; convert to RGB for display.

We are going to look hard at the middle three, because they are where the interesting ideas live -- and I will be honest about the entropy decoder being the fiddliest part to get fully right.

Walking the marker stream

A JPEG file is a stream of markers. Every marker is two bytes: 0xFF followed by a marker code that is not 0x00 and not 0xFF. The file opens with SOI (0xFFD8, Start Of Image) and ends with EOI (0xFFD9, End Of Image). Most markers introduce a segment: after the two marker bytes comes a big-endian u16 length (which includes its own two bytes), then that many bytes of payload. A few markers -- SOI, EOI, and the restart markers -- stand alone with no length. Sounds tidy, and it mostly is, with one wrinkle we will hit in a moment. Let me model the markers we care about as an enum and write a small reader:

const std = @import("std");

pub const JpegError = error{
    BadMagic,
    Truncated,
    UnsupportedFormat,
    InvalidData,
    MissingTables,
};

pub const Marker = enum(u16) {
    soi = 0xFFD8, // start of image
    eoi = 0xFFD9, // end of image
    sof0 = 0xFFC0, // baseline DCT frame header
    dht = 0xFFC4, // define Huffman table(s)
    dqt = 0xFFDB, // define quantization table(s)
    sos = 0xFFDA, // start of scan (compressed data follows)
    app0 = 0xFFE0, // JFIF application segment
    com = 0xFFFE, // comment
    _, // everything else we skip
};

pub const SegmentReader = struct {
    bytes: []const u8,
    pos: usize,

    pub fn init(bytes: []const u8) JpegError!SegmentReader {
        if (bytes.len < 2 or bytes[0] != 0xFF or bytes[1] != 0xD8)
            return JpegError.BadMagic; // must start with SOI
        return .{ .bytes = bytes, .pos = 2 };
    }

    // Return the next marker code and (for segment markers) its payload slice.
    pub fn next(self: *SegmentReader) JpegError!?struct { marker: u16, payload: []const u8 } {
        // Markers may be preceded by fill bytes; skip any run of 0xFF then read the code.
        while (self.pos < self.bytes.len and self.bytes[self.pos] != 0xFF) self.pos += 1;
        while (self.pos < self.bytes.len and self.bytes[self.pos] == 0xFF) self.pos += 1;
        if (self.pos >= self.bytes.len) return null;

        const code: u16 = 0xFF00 | @as(u16, self.bytes[self.pos]);
        self.pos += 1;
        // SOI, EOI and restart markers carry no length.
        if (code == 0xFFD9 or code == 0xFFD8) return .{ .marker = code, .payload = &.{} };

        if (self.pos + 2 > self.bytes.len) return JpegError.Truncated;
        const len = std.mem.readInt(u16, self.bytes[self.pos..][0..2], .big);
        if (len < 2) return JpegError.InvalidData;
        const end = self.pos + len;
        if (end > self.bytes.len) return JpegError.Truncated;
        const payload = self.bytes[self.pos + 2 .. end]; // length includes its own 2 bytes
        self.pos = end;
        return .{ .marker = code, .payload = payload };
    }
};

The _ in the enum is Zig's non-exhaustive enum marker (episode 6) -- it lets me name the handful of markers I care about while still accepting the dozens I do not, so a stray APP14 or DRI becomes a value I can match on else in stead of an illegal-enum panic. The length-includes-itself detail (len < 2 is malformed, real payload is len - 2 bytes) is the kind of off-by-two that a decoder written from a half-remembered spec always gets wrong the first time; making it explicit in one place means we only get it wrong once. And every read that could run off the end returns Truncated, because -- same sermon as last episode -- an untrusted file is a hostile file.

Quantization and Huffman tables

Two segment types carry the tables the scan will need. DQT defines one or more 8x8 quantization tables, each 64 bytes (for 8-bit precision), stored in zig-zag order (more on that ordering in a second). DHT defines Huffman tables: 16 bytes giving how many codes exist of each bit-length 1 through 16, followed by the symbol values in code order -- the canonical Huffman layout, the same idea you would meet in DEFLATE. Here is DQT decoded into a fixed array, and the skeleton of a Huffman table:

const std = @import("std");

pub const QuantTable = [64]u16; // dequant multipliers, in natural (de-zigzagged) order

// Parse one DQT segment (it may define several tables back to back).
fn parseDqt(payload: []const u8, tables: *[4]?QuantTable) JpegError!void {
    var i: usize = 0;
    while (i < payload.len) {
        const pq_tq = payload[i]; // high nibble = precision, low nibble = table id
        const precision = pq_tq >> 4;
        const id = pq_tq & 0x0F;
        if (id >= 4) return JpegError.InvalidData;
        i += 1;
        if (precision != 0) return JpegError.UnsupportedFormat; // 8-bit tables only here

        if (i + 64 > payload.len) return JpegError.Truncated;
        var t: QuantTable = undefined;
        // Stored in zig-zag order; place each entry at its natural grid position.
        for (0..64) |k| t[zigzag[k]] = payload[i + k];
        tables[id] = t;
        i += 64;
    }
}

pub const HuffTable = struct {
    // Canonical decode helpers: for each code length, the first code and the
    // index into `symbols` where that length's symbols begin.
    counts: [17]u8 = [_]u8{0} ** 17, // counts[len] = number of codes of that bit length
    symbols: [256]u8 = undefined,
    symbol_count: usize = 0,
};

Notice the de-zig-zag happening right at parse time: t[zigzag[k]] = payload[i + k] scatters the 64 stored bytes into their true 2D grid positions, so downstream code can index the table as a plain 8x8 array. Doing that translation once, at the boundary, keeps every later stage from having to think about the wire order. The high-nibble/low-nibble unpack of pq_tq is classic episode-17 bit work -- one byte carrying two fields, split with a shift and a mask. Guarding id >= 4 matters because the id indexes a fixed four-slot array; an out-of-range table id in a crafted file must be an error, not an array overrun.

The zig-zag order

Why "zig-zag" at all? After the DCT, the 64 coefficients of a block are ordered by frequency -- the top-left is the average (the "DC" term), and moving right and down increases horizontal and vertical frequency. The encoder wants all the near-zero high-frequency coefficients to end up adjacent so a run-length step can collapse them. Reading the 8x8 grid in a diagonal zig-zag does exactly that: it visits coefficients roughly in order of increasing total frequency, so the long tail of zeros lands at the end. We need the same table to reverse it:

const std = @import("std");

// Maps zig-zag position (0..63) -> natural row-major position in an 8x8 block.
pub const zigzag = [64]u8{
    0,  1,  8,  16, 9,  2,  3,  10,
    17, 24, 32, 25, 18, 11, 4,  5,
    12, 19, 26, 33, 40, 48, 41, 34,
    27, 20, 13, 6,  7,  14, 21, 28,
    35, 42, 49, 56, 57, 50, 43, 36,
    29, 22, 15, 23, 30, 37, 44, 51,
    58, 59, 52, 45, 38, 31, 39, 46,
    53, 60, 61, 54, 47, 55, 62, 63,
};

test "zigzag is a permutation of 0..63" {
    var seen = [_]bool{false} ** 64;
    for (zigzag) |p| {
        try std.testing.expect(!seen[p]); // no position visited twice
        seen[p] = true;
    }
    for (seen) |s| try std.testing.expect(s); // every position visited
}

That test is the cheapest possible correctness guard on a lookup table: a zig-zag map must be a permutation -- every one of the 64 grid positions appears exactly once. If a typo duplicates an index (which is desperately easy in a hand-typed 64-entry table), the test fails immediately in stead of a subtly-scrambled image showing up three stages later. This is episode 12's TDD instinct applied to data, not code: verify the invariant of the table itself.

Dequantize and the inverse DCT

Now the mathematical heart. Each 8x8 block arrives from the entropy decoder as 64 quantized coefficients. Dequantizing is a single element-wise multiply by the quant table -- undoing the encoder's divide-and-round (the round is where the information was lost; we cannot get it back, only undo the scaling). Then the inverse Discrete Cosine Transform turns those 64 frequency coefficients into 64 spatial samples. The fast DCTs real decoders use (AAN, Loeffler) are clever factorizations; for understanding, the honest thing is the straight double sum -- slow, but it is the definition, and you can read every term:

const std = @import("std");

// Dequantize in place: coefficient[i] *= quant[i].
fn dequantize(block: *[64]i32, quant: QuantTable) void {
    for (block, 0..) |*c, i| c.* *= @as(i32, quant[i]);
}

// Naive separable inverse DCT of an 8x8 block. `in` is dequantized coefficients in
// natural order; `out` receives spatial samples, shifted back to the 0..255 range.
fn idct8x8(in: *const [64]i32, out: *[64]u8) void {
    var tmp: [64]f32 = undefined;
    for (0..8) |y| {
        for (0..8) |x| {
            var sum: f32 = 0;
            for (0..8) |v| {
                for (0..8) |u| {
                    const cu: f32 = if (u == 0) std.math.sqrt1_2 else 1.0;
                    const cv: f32 = if (v == 0) std.math.sqrt1_2 else 1.0;
                    const coeff: f32 = @floatFromInt(in[v * 8 + u]);
                    const cx = std.math.cos(@as(f32, @floatFromInt(2 * x + 1)) * @as(f32, @floatFromInt(u)) * std.math.pi / 16.0);
                    const cy = std.math.cos(@as(f32, @floatFromInt(2 * y + 1)) * @as(f32, @floatFromInt(v)) * std.math.pi / 16.0);
                    sum += cu * cv * coeff * cx * cy;
                }
            }
            tmp[y * 8 + x] = sum / 4.0;
        }
    }
    // JPEG samples were level-shifted by -128 before the forward DCT; undo that and clamp.
    for (tmp, 0..) |s, i| {
        const shifted = std.math.clamp(s + 128.0, 0.0, 255.0);
        out[i] = @intFromFloat(@round(shifted));
    }
}

The cu/cv factors (the 1/sqrt(2) normalization on the DC row and column) and that final / 4.0 are the normalization constants that make the inverse the true inverse of the forward transform -- get them wrong and the image comes out too dark or too bright by a constant factor. The + 128 level shift is a JPEG-specific quirk: the encoder subtracted 128 from every sample before transforming (to center the data around zero), so we add it back and clamp into 0..255. I have written this as the four-nested-loop definition on purpose. It is O(n^4) per block and no one ships it -- but it is readable, and once you have this working and tested, swapping in a fast separable IDCT (two passes of 1D transforms) is a well-defined optimization with a reference to check against. That is the right order: correct-and-slow first, fast-and-verified second (episode 34, always).

From YCbCr to RGB

The IDCT gives us spatial samples, but in JPEG's native color space: YCbCr, where Y is luma (brightness) and Cb/Cr are the blue- and red-difference chroma channels. This is exactly the terrain episode 165 prepared us for. The conversion back to RGB is the standard JFIF matrix, and here is where our decoded blocks finally become the Rgba pixels our framebuffer speaks:

const std = @import("std");
pub const Rgba = packed struct { r: u8, g: u8, b: u8, a: u8 = 255 };

fn clampByte(v: f32) u8 {
    return @intFromFloat(@round(std.math.clamp(v, 0.0, 255.0)));
}

// JFIF full-range YCbCr -> RGB. Cb/Cr are stored biased by +128.
fn ycbcrToRgb(y: u8, cb: u8, cr: u8) Rgba {
    const yf: f32 = @floatFromInt(y);
    const cbf: f32 = @as(f32, @floatFromInt(cb)) - 128.0;
    const crf: f32 = @as(f32, @floatFromInt(cr)) - 128.0;
    return .{
        .r = clampByte(yf + 1.402 * crf),
        .g = clampByte(yf - 0.344136 * cbf - 0.714136 * crf),
        .b = clampByte(yf + 1.772 * cbf),
    };
}

test "neutral chroma is pure gray" {
    // Cb = Cr = 128 means "no color"; R = G = B = Y for any luma.
    const px = ycbcrToRgb(200, 128, 128);
    try std.testing.expect(px.r == px.g and px.g == px.b);
    try std.testing.expectEqual(@as(u8, 200), px.r);
}

The test pins the property that makes the whole color model intuitive: when both chroma channels sit at their neutral 128, there is no color information, so every channel collapses to the luma value and you get a pure gray. Those magic constants (1.402, 0.344136, and friends) are the inverse of the RGB-to-YCbCr matrix JFIF specifies -- I would keep them in one named function exactly like this, because a stray digit here tints the entire image and it is maddening to debug from a photo. One subtlety I am glossing for the "basics": most JPEGs store chroma at half resolution (the famous 4:2:0 subsampling), so a real decoder upsamples Cb/Cr -- doubling them in each direction -- before this conversion. Wiring that up is one of the exercises.

Testing a lossy codec

You cannot assert "decoded pixel equals original pixel" for a lossy format -- the whole point is that they differ. So the testing strategy shifts. Three things work well, in increasing scope. First, test the pure, exact pieces against known vectors: the zig-zag permutation, the Huffman table build, the YCbCr matrix on neutral chroma -- all of those are lossless and deterministic, so exact assertions are fair game (we did two above). Second, test the DCT pair as a round-trip within a tolerance: a forward DCT followed by our inverse should return the original block to within a small epsilon, since only floating-point error separates them (not quantization). Third, for the whole decoder, compare against a reference decoder with a similarity metric like PSNR, asserting "close enough", not "identical":

const std = @import("std");

// Peak signal-to-noise ratio between two equally-sized byte buffers, in decibels.
// Higher is better; a faithful baseline JPEG decode against a reference is typically 30-50 dB.
fn psnr(a: []const u8, b: []const u8) f64 {
    std.debug.assert(a.len == b.len);
    var mse: f64 = 0;
    for (a, b) |x, y| {
        const d: f64 = @as(f64, @floatFromInt(x)) - @as(f64, @floatFromInt(y));
        mse += d * d;
    }
    mse /= @floatFromInt(a.len);
    if (mse == 0) return std.math.inf(f64); // identical
    return 10.0 * std.math.log10(255.0 * 255.0 / mse);
}

test "psnr of a buffer against itself is infinite, against noise is finite" {
    const img = [_]u8{ 10, 20, 30, 40 };
    try std.testing.expect(std.math.isInf(psnr(&img, &img)));
    const noisy = [_]u8{ 12, 18, 33, 37 };
    try std.testing.expect(psnr(&img, &noisy) < 60.0);
}

PSNR is the honest tool for lossy work: it gives you a single number that says "how far did my decode drift from the reference", and you assert a floor (say, "at least 35 dB against libjpeg's output on this fixture"). It is not a perfect perceptual measure -- two images with the same PSNR can look differently wrong -- but as a regression guard it is exactly right: if a refactor drops the PSNR from 42 to 12, you broke something, loudly. Testing the invariant beats testing the bytes, once again.

Performance: where the time actually goes

Profile a JPEG decoder (episode 34, measure before you guess) and the cost lands in two places, neither of them the marker parsing. First, the entropy decode -- pulling bits out of the Huffman-coded scan, one variable-length code at a time -- because it is inherently serial and bit-at-a-time. Real decoders speed it with multi-bit lookup tables that decode a whole short code in one array index in stead of walking bit by bit. Second, the inverse DCT, which runs once per 8x8 block -- and for a multi-megapixel photo that is a lot of blocks. Our naive O(n^4) IDCT is the obvious target; the fast separable transforms drop it to two passes of 1D DCTs (O(n^3) and with far smaller constants), and the AAN factorization famously gets an 8-point 1D DCT down to a handful of multiplies. A few honest levers, in episode-34 priority order:

  • Replace the naive IDCT with a separable one the moment a profile confirms it dominates. Two 1D passes (rows then columns) is the single biggest win and has a clear reference output to check against.
  • Table-drive the Huffman decode. An 8-bit peek that resolves most codes in one lookup beats a per-bit walk, at the cost of a small precomputed table -- the same trick DEFLATE decoders use.
  • Precompute the IDCT cosines. Those cos(...) calls in the inner loop are constant across every block; hoist them into a comptime-built 8x8 table (episode 9) so the hot path is multiplies and adds, no transcendental functions.
  • Vectorize the dequantize and the 1D DCT passes. Element-wise multiply of 64 coefficients by 64 quant values is a textbook @Vector case (episode 19).

The meta-point does not change: for one thumbnail, none of this matters and the naive version is fine; for a photo gallery, the IDCT and entropy decode are the whole budget. Measure, then sharpen the knife that is actually dull.

The same job in C, Rust and Go

The stages are identical in every language -- markers, tables, entropy decode, dequantize, IDCT, color convert -- so what differs is, once more, the posture toward memory, errors, and whether you write it yourself at all. In C, essentially nobody hand-writes a baseline decoder: you link libjpeg (or the drop-in faster libjpeg-turbo), which hands you a decompression struct you drive through a setup/scanline/teardown dance, with error handling bolted on through a setjmp/longjmp escape hatch that is a genuine wart:

// C with libjpeg: real decoders reach for the library. Note the setjmp error hatch --
// there are no exceptions and no error unions, so libjpeg longjmps out of an error handler.
struct jpeg_decompress_struct cinfo;
struct my_error_mgr jerr; // wraps jpeg_error_mgr + a jmp_buf
cinfo.err = jpeg_std_error(&jerr.pub);
if (setjmp(jerr.setjmp_buffer)) { jpeg_destroy_decompress(&cinfo); return NULL; }
jpeg_create_decompress(&cinfo);
jpeg_mem_src(&cinfo, data, len);
jpeg_read_header(&cinfo, TRUE);
jpeg_start_decompress(&cinfo); /* then read scanlines row by row */

Rust reaches for the image crate or the dedicated jpeg-decoder, and the error story is the pleasant one -- a Result you ?-propagate, no longjmp in sight, and the borrow checker keeping the coefficient buffers honest:

// Rust: the jpeg-decoder crate. Errors are values, propagated with `?`.
use jpeg_decoder::Decoder;
fn decode(bytes: &[u8]) -> Result<Vec<u8>, jpeg_decoder::Error> {
    let mut decoder = Decoder::new(bytes);
    let pixels = decoder.decode()?;        // Vec, interleaved by pixel format
    let _info = decoder.info().unwrap();   // dimensions, pixel format
    Ok(pixels)
}

Go ships image/jpeg in its standard library, so it is a one-liner with no third-party dependency at all -- Go's batteries-included philosophy exactly as we saw with image/png last episode:

// Go: standard library, no dependency. png last episode, jpeg this one -- same shape.
func decodeJpeg(r io.Reader) (image.Image, error) {
    img, err := jpeg.Decode(r) // returns an image.Image and an error
    if err != nil {
        return nil, err
    }
    return img, nil
}

The pattern is the same one PNG showed, only more so: Go ships a decoder, Rust and C reach for a well-worn crate or library, and the ecosystems agree that nobody should write a JPEG decoder for production -- the format is a minefield of edge cases (progressive scans, restart intervals, arithmetic coding, exotic subsampling) that a library has already survived. Where does Zig's own decoder sit? For real work you would reach for a C library via the zero-cost interop from episodes 27 and 28, or a community package. But doing the baseline once by hand, as we just walked, is what turns "JPEG is magic" into "JPEG is a DCT, a couple of tables, and a bitstream" -- and Zig's explicitness (named markers, error unions on every truncation, tables de-zigzagged at the boundary) makes that walk unusually legible.

Exercises

  1. Finish the DHT parser and Huffman decode. Fill in parseDqt's sibling parseDht: read the 16 length-counts, then the symbols, and build a canonical decode structure. Then write a BitReader over the scan data that skips the 0xFF 0x00 byte-stuffing (JPEG escapes a literal 0xFF in the entropy stream as 0xFF 0x00) and decodes one Huffman symbol by walking bit lengths. Test it against a tiny hand-built table where you know the codes.

  2. Add 4:2:0 chroma upsampling. Our ycbcrToRgb assumes Y, Cb and Cr all share a resolution, but baseline JPEGs usually store chroma at half width and half height. Parse the SOF0 component sampling factors, and before color conversion, upsample the Cb/Cr planes by nearest-neighbour doubling (then, as a stretch, try bilinear and compare the PSNR against a reference).

  3. Build a comptime IDCT cosine table. Replace the std.math.cos calls in the inner loop with a lookup into an 8x8 (or 8x8x8x8) table computed at compile time with a comptime block (episode 9). Write a test that the table-driven IDCT matches the naive one to within 1e-4 on a few random coefficient blocks, then note the speedup you would expect and why.

What we learned

  • JPEG is lossy and works in the frequency domain -- it transforms 8x8 blocks with a DCT and throws away the high-frequency coefficients the eye barely sees, which is why it crushes photos and mangles sharp text;
  • A JPEG file is a marker-and-segment stream (0xFF + code, big-endian length that includes itself), opened by SOI and closed by EOI, and Zig's non-exhaustive enum lets us name the markers we handle while skipping the rest as explicit errors, never panics;
  • The DQT and DHT segments carry the quantization and Huffman tables, and de-zig-zagging the quant entries at parse time keeps every later stage indexing a plain 8x8 grid;
  • The decode pipeline is entropy decode, dequantize, inverse DCT, upsample, YCbCr to RGB -- and the naive four-loop IDCT plus the JFIF color matrix are readable, testable definitions you can later swap for fast, verified versions (episode 34's correct-then-fast discipline);
  • Testing a lossy codec means exact assertions on the lossless pieces (the zig-zag permutation, the neutral-chroma gray), tolerance round-trips on the DCT, and a PSNR floor against a reference for the whole decode -- test the invariant, not the exact bytes;
  • C, Rust and Go all agree nobody ships a hand-written JPEG decoder -- Go has it in the stdlib, Rust and C reach for a crate or libjpeg -- but walking the baseline once by hand turns the format from magic into machinery.

That closes the image-formats arc that started all the way back at episode 44: we can read PPM/BMP, decode PNG losslessly, and now understand a lossy JPEG stage by stage, all landing as Rgba in the framebuffer we built. We have spent a long stretch making the machine show things -- pixels, shapes, glyphs, decoded photos. There is a whole other sense we have not touched yet, one that is also just numbers in a buffer if you look at it right, and it is where we head next. Plenty still to build.

Bedankt voor het lezen, tot de volgende keer! ;-)

scipio@scipio

stem
stemsocial
steemstem
zig
programming
via hiveblog
This post was published via hiveblog. Ecency is not the originator or editor of this content and displays it for discovery purposes only.
12
Learn Zig Series (#168) - JPEG Decoder Basics | Ecency