Learn Zig Series (#164) - TrueType Parsing

scipio(71)
Published in
StemSocial
Words
3396
Reading
16 min
Listen
Play
2h

Learn Zig Series (#164) - TrueType Parsing

zig.png

What will I learn?

  • Why a bitmap font hits a wall the moment you want more than one size, and why the fix is to read a real font file's outlines;
  • The shape of a TrueType file: an offset table, a directory of named tables, and the tables (head, maxp, loca, glyf, cmap) you need to find a glyph;
  • A big-endian byte reader that pulls u16, u32 and i16 from a slice, bounds-checked, because every number in a font file is big-endian;
  • How to find a table by its four-byte tag, parse head for unitsPerEm and the loca format, and maxp for the glyph count;
  • How loca turns a glyph id into a byte range in glyf, and how a glyph header gives the contour count (negative means composite) and bounding box;
  • A first look at cmap, with format 0 built and tested and format 4 explained;
  • How C, Rust and Go write the same big-endian read and table scan.

Requirements

  • A working modern computer running macOS, Windows or Ubuntu;
  • An installed Zig 0.14+ distribution (download from ziglang.org) -- the code here is written and tested against Zig 0.16;
  • The bitmap-font drawGlyph, Font and drawText from episode 163, and Framebuffer/Rgba from episode 156;
  • Comfort with slices (episode 5), optionals and the if (x) |y| capture (episode 4), and error unions (episode 4);
  • The ambition to learn Zig programming.

Difficulty

  • Advanced

Curriculum (of the Learn Zig Series):

Learn Zig Series (#164) - TrueType Parsing

Last episode built a font that draws crisp text at exactly the one size we hand-drew it. Zoom in and it becomes a staircase of fat pixels. The escape is to store the outline of each letter -- lines and curves -- in a real font file, and rasterize it fresh at any size. But first we have to open that file, and a .ttf is a dense binary container of tables, every number big-endian, every table found through an offset. Today is the parsing adventure: crack the file open, walk its directory, and read down to a single glyph's outline header. No drawing yet. First, three loose ends from last time. Here we go!

Solutions to Episode 163 Exercises

Exercise 1 -- a proportional font. Give each glyph its own width with a parallel advances slice and an advanceFor method, so a narrow letter need not eat the same space as a wide one. drawText changes by exactly one line -- the cursor advances by advanceFor(c) in stead of a constant:

const std = @import("std");
pub const Rgba = packed struct { r: u8, g: u8, b: u8, a: u8 = 255 };
pub const Framebuffer = struct {
    pixels: []Rgba,
    width: usize,
    height: usize,
    pub fn setPixel(self: *Framebuffer, x: i64, y: i64, color: Rgba) void {
        if (x < 0 or y < 0) return;
        const ux: usize = @intCast(x);
        const uy: usize = @intCast(y);
        if (ux >= self.width or uy >= self.height) return;
        self.pixels[uy * self.width + ux] = color;
    }
    pub fn getPixel(self: Framebuffer, x: i64, y: i64) ?Rgba {
        if (x < 0 or y < 0) return null;
        const ux: usize = @intCast(x);
        const uy: usize = @intCast(y);
        if (ux >= self.width or uy >= self.height) return null;
        return self.pixels[uy * self.width + ux];
    }
};
pub const Glyph8x8 = [8]u8;
pub fn drawGlyph(fb: *Framebuffer, glyph: Glyph8x8, x: i64, y: i64, ink: Rgba) void {
    var row: usize = 0;
    while (row < 8) : (row += 1) {
        const bits = glyph[row];
        var col: usize = 0;
        while (col < 8) : (col += 1) {
            const shift: u3 = @intCast(7 - col);
            if (((bits >> shift) & 1) == 1)
                fb.setPixel(x + @as(i64, @intCast(col)), y + @as(i64, @intCast(row)), ink);
        }
    }
}

// A proportional font gives every glyph its own width via a parallel advances slice,
// so a narrow letter need not eat the same space as a wide one. drawText changes by
// exactly one line: it advances the cursor by advanceFor(c) in stead of a constant.
pub const Font = struct {
    glyphs: []const Glyph8x8,
    advances: []const u8,
    first: u8,
    line_height: i64 = 8,
    fn glyphFor(self: Font, c: u8) ?Glyph8x8 {
        if (c < self.first) return null;
        const idx: usize = c - self.first;
        if (idx >= self.glyphs.len) return null;
        return self.glyphs[idx];
    }
    fn advanceFor(self: Font, c: u8) i64 {
        if (c < self.first) return self.line_height;
        const idx: usize = c - self.first;
        if (idx >= self.advances.len) return self.line_height;
        return self.advances[idx];
    }
};
pub fn drawText(fb: *Framebuffer, font: Font, text: []const u8, x: i64, y: i64, ink: Rgba) void {
    var cx = x;
    var cy = y;
    for (text) |c| {
        if (c == '\n') {
            cx = x;
            cy += font.line_height;
            continue;
        }
        if (font.glyphFor(c)) |g| drawGlyph(fb, g, cx, cy, ink);
        cx += font.advanceFor(c);
    }
}

// The table is indexed by (c - first), so entries must be consecutive characters.
// Two glyphs lit only at their top-left pixel, so a rendered dot marks exactly where
// each glyph landed -- unambiguous for testing the cursor math. 'i' is given width 3,
// its neighbour 'j' width 7.
const glyph_i: Glyph8x8 = .{ 0b10000000, 0, 0, 0, 0, 0, 0, 0 };
const glyph_j: Glyph8x8 = .{ 0b10000000, 0, 0, 0, 0, 0, 0, 0 };

test "a proportional font advances by each glyph's own width, not a fixed step" {
    const font = Font{ .glyphs = &[_]Glyph8x8{ glyph_i, glyph_j }, .advances = &[_]u8{ 3, 7 }, .first = 'i' };
    try std.testing.expectEqual(@as(i64, 3), font.advanceFor('i'));
    try std.testing.expectEqual(@as(i64, 7), font.advanceFor('j'));

    var pixels: [32 * 8]Rgba = undefined;
    var fb = Framebuffer{ .pixels = &pixels, .width = 32, .height = 8 };
    @memset(fb.pixels, .{ .r = 0, .g = 0, .b = 0 });
    const white = Rgba{ .r = 255, .g = 255, .b = 255 };
    drawText(&fb, font, "ij", 0, 0, white);

    try std.testing.expectEqual(@as(u8, 255), fb.getPixel(0, 0).?.r); // 'i' drew at x=0
    try std.testing.expectEqual(@as(u8, 255), fb.getPixel(3, 0).?.r); // 'j' started at x=3 (the 'i' advance)
    try std.testing.expectEqual(@as(u8, 0), fb.getPixel(8, 0).?.r); // NOT at a fixed step of 8
}

That single call is the whole difference between a monospace and a proportional font. Nota bene the sane default for characters outside the table, so an unknown byte still moves the cursor.

Exercise 2 -- word wrapping. Break at the last space before a maximum width, and -- the sneaky part -- a single word longer than the limit must still make progress, never loop forever. Before drawing each word I check whether it would run past the limit and we are not already at the line start; only then wrap:

const std = @import("std");
pub const Rgba = packed struct { r: u8, g: u8, b: u8, a: u8 = 255 };
pub const Framebuffer = struct {
    pixels: []Rgba,
    width: usize,
    height: usize,
    pub fn setPixel(self: *Framebuffer, x: i64, y: i64, color: Rgba) void {
        if (x < 0 or y < 0) return;
        const ux: usize = @intCast(x);
        const uy: usize = @intCast(y);
        if (ux >= self.width or uy >= self.height) return;
        self.pixels[uy * self.width + ux] = color;
    }
};
pub const Glyph8x8 = [8]u8;
pub fn drawGlyph(fb: *Framebuffer, glyph: Glyph8x8, x: i64, y: i64, ink: Rgba) void {
    var row: usize = 0;
    while (row < 8) : (row += 1) {
        const bits = glyph[row];
        var col: usize = 0;
        while (col < 8) : (col += 1) {
            const shift: u3 = @intCast(7 - col);
            if (((bits >> shift) & 1) == 1)
                fb.setPixel(x + @as(i64, @intCast(col)), y + @as(i64, @intCast(row)), ink);
        }
    }
}
pub const Font = struct {
    glyphs: []const Glyph8x8,
    first: u8,
    advance: i64 = 8,
    line_height: i64 = 8,
    fn glyphFor(self: Font, c: u8) ?Glyph8x8 {
        if (c < self.first) return null;
        const idx: usize = c - self.first;
        if (idx >= self.glyphs.len) return null;
        return self.glyphs[idx];
    }
};

// Break to a new line at the last space before max_width in stead of mid-word. A
// single word longer than the limit is still drawn and we move on -- progress over
// perfection, so a giant token can never trap the layout in an infinite loop. Returns
// the final pen position so callers (and tests) can see where the text ended up.
pub fn drawTextWrapped(fb: *Framebuffer, font: Font, text: []const u8, x: i64, y: i64, max_width: i64, ink: Rgba) [2]i64 {
    var cx = x;
    var cy = y;
    var it = std.mem.splitScalar(u8, text, ' ');
    while (it.next()) |word| {
        const w: i64 = @as(i64, @intCast(word.len)) * font.advance;
        if (cx != x and cx + w > x + max_width) {
            cx = x;
            cy += font.line_height;
        }
        for (word) |c| {
            if (font.glyphFor(c)) |g| drawGlyph(fb, g, cx, cy, ink);
            cx += font.advance;
        }
        cx += font.advance; // the space that followed the word
    }
    return .{ cx, cy };
}

const glyph_a: Glyph8x8 = .{ 0, 0, 0, 0, 0, 0, 0, 0 };

test "wrapping breaks at a space, and a too-long word still makes progress" {
    const font = Font{ .glyphs = &[_]Glyph8x8{glyph_a}, .first = 'a' };
    var pixels: [64 * 64]Rgba = undefined;
    var fb = Framebuffer{ .pixels = &pixels, .width = 64, .height = 64 };
    @memset(fb.pixels, .{ .r = 0, .g = 0, .b = 0 });
    const ink = Rgba{ .r = 255, .g = 255, .b = 255 };

    // limit 24 px (3 glyphs): "aaa" fits on line 1, the second "aaa" wraps to line 2.
    const end = drawTextWrapped(&fb, font, "aaa aaa", 0, 0, 24, ink);
    try std.testing.expectEqual(@as(i64, 8), end[1]); // cy advanced one line: it wrapped

    // a 10-glyph word with a 24 px limit: drawn anyway, no infinite loop, cursor moved on.
    const end2 = drawTextWrapped(&fb, font, "aaaaaaaaaa", 0, 0, 24, ink);
    try std.testing.expectEqual(@as(i64, 0), end2[1]); // stayed on the first line
    try std.testing.expect(end2[0] > 24); // it made progress past the limit
}

The cx != x guard is the trick. Without it a word wider than the whole line would wrap, land at the start, still not fit, wrap again... forever. With it, a too-long word is simply drawn as-is and we move on -- ugly, but progress beats an infinite loop. Which would you rather ship?

Exercise 3 -- a background box. An optional background color so each glyph cell is filled first and the ink stamped on top, for readable text over a busy image. A cleared bit paints the background when it is set, and is skipped when it is null -- exactly last episode's transparency:

const std = @import("std");
pub const Rgba = packed struct { r: u8, g: u8, b: u8, a: u8 = 255 };
pub const Framebuffer = struct {
    pixels: []Rgba,
    width: usize,
    height: usize,
    pub fn setPixel(self: *Framebuffer, x: i64, y: i64, color: Rgba) void {
        if (x < 0 or y < 0) return;
        const ux: usize = @intCast(x);
        const uy: usize = @intCast(y);
        if (ux >= self.width or uy >= self.height) return;
        self.pixels[uy * self.width + ux] = color;
    }
    pub fn getPixel(self: Framebuffer, x: i64, y: i64) ?Rgba {
        if (x < 0 or y < 0) return null;
        const ux: usize = @intCast(x);
        const uy: usize = @intCast(y);
        if (ux >= self.width or uy >= self.height) return null;
        return self.pixels[uy * self.width + ux];
    }
};
pub const Glyph8x8 = [8]u8;

// An optional background: when bg is non-null, a cleared bit paints the background
// color (an opaque box behind the text); when bg is null we touch only the ink pixels
// -- exactly the color-key transparency from episode 163. A set bit always paints ink.
pub fn drawGlyphBg(fb: *Framebuffer, glyph: Glyph8x8, x: i64, y: i64, ink: Rgba, bg: ?Rgba) void {
    var row: usize = 0;
    while (row < 8) : (row += 1) {
        const bits = glyph[row];
        var col: usize = 0;
        while (col < 8) : (col += 1) {
            const shift: u3 = @intCast(7 - col);
            const px = x + @as(i64, @intCast(col));
            const py = y + @as(i64, @intCast(row));
            if (((bits >> shift) & 1) == 1) {
                fb.setPixel(px, py, ink);
            } else if (bg) |b| {
                fb.setPixel(px, py, b);
            }
        }
    }
}

const glyph_H: Glyph8x8 = .{ 0b01000010, 0b01000010, 0b01000010, 0b01111110, 0b01000010, 0b01000010, 0b01000010, 0b00000000 };

test "a background box fills cleared bits when set, and leaves them untouched when null" {
    const white = Rgba{ .r = 255, .g = 255, .b = 255 };
    const red = Rgba{ .r = 255, .g = 0, .b = 0 };

    var box: [8 * 8]Rgba = undefined;
    var fb_box = Framebuffer{ .pixels = &box, .width = 8, .height = 8 };
    @memset(fb_box.pixels, .{ .r = 0, .g = 0, .b = 0 });
    drawGlyphBg(&fb_box, glyph_H, 0, 0, white, red);
    try std.testing.expectEqual(@as(u8, 255), fb_box.getPixel(1, 0).?.g); // set bit -> white ink
    try std.testing.expectEqual(@as(u8, 0), fb_box.getPixel(0, 0).?.g); // cleared bit -> red bg (g=0)
    try std.testing.expectEqual(@as(u8, 255), fb_box.getPixel(0, 0).?.r); // red channel of the bg is on

    var clear: [8 * 8]Rgba = undefined;
    var fb_clear = Framebuffer{ .pixels = &clear, .width = 8, .height = 8 };
    @memset(fb_clear.pixels, .{ .r = 0, .g = 200, .b = 0 }); // a green background image
    drawGlyphBg(&fb_clear, glyph_H, 0, 0, white, null);
    try std.testing.expectEqual(@as(u8, 255), fb_clear.getPixel(1, 0).?.r); // set bit -> white ink
    try std.testing.expectEqual(@as(u8, 200), fb_clear.getPixel(0, 0).?.g); // cleared bit -> untouched green
}

The else if (bg) |b| is the entire feature: the optional decides, at the call site, whether a cleared bit is an opaque box or transparent. Right, exercises done. Time to open a font file.

What is actually inside a .ttf

A TrueType font is a container. The file opens with an offset table: a four-byte version (0x00010000 for TrueType outlines, or OTTO for the CFF flavour), the number of tables, then three fields we can ignore. Then the table directory: one 16-byte record per table, each with a four-byte tag (head, glyf, cmap), a checksum, the table's offset from the start of the file, and its length. You reach any table by finding its tag and jumping to its offset.

The one rule that colors every line: every multi-byte number is big-endian. So we need a reader that pulls integers from a byte slice, big-endian, and refuses to run off the end -- a font file is untrusted input, and every offset is a chance to read past the buffer:

const std = @import("std");

// TrueType stores every multi-byte number big-endian. This reader wraps a byte
// slice and pulls fixed-width integers from an offset, bounds-checked so a truncated
// or hostile file returns an error in stead of reading past the end of the buffer.
pub const ParseError = error{ OutOfBounds, BadMagic, TableMissing, BadLocaFormat };

pub const ByteReader = struct {
    data: []const u8,

    pub fn u16At(self: ByteReader, off: usize) ParseError!u16 {
        if (off + 2 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(u16, self.data[off..][0..2], .big);
    }
    pub fn u32At(self: ByteReader, off: usize) ParseError!u32 {
        if (off + 4 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(u32, self.data[off..][0..4], .big);
    }
    pub fn i16At(self: ByteReader, off: usize) ParseError!i16 {
        if (off + 2 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(i16, self.data[off..][0..2], .big);
    }
};

test "the reader pulls big-endian integers and refuses to run off the end" {
    const bytes = [_]u8{ 0x00, 0x01, 0x00, 0x00, 0x03, 0xE8, 0xFF, 0xFF };
    const r = ByteReader{ .data = &bytes };
    try std.testing.expectEqual(@as(u32, 0x00010000), try r.u32At(0)); // sfnt version 1.0
    try std.testing.expectEqual(@as(u16, 1000), try r.u16At(4)); // 0x03E8 = 1000
    try std.testing.expectEqual(@as(i16, -1), try r.i16At(6)); // 0xFFFF = -1 signed
    try std.testing.expectError(error.OutOfBounds, r.u32At(6)); // only 2 bytes remain
}

std.mem.readInt does the byte-order work; .big is the point, and data[off..][0..2] hands it exactly two bytes so the read width is checked at compile time. The bounds check is not paranoia -- font files come off the network, and "read a u32 near the end" is how a naive parser walks off its buffer. Here the worst case is a clean error.OutOfBounds.

The offset table and finding a table by tag

The offset table is four lines, and the directory scan is a loop: walk the 16-byte records, compare each tag, hand back the offset and length. I reject a bad version early, because that almost always means "this is not a font":

const std = @import("std");
pub const ParseError = error{ OutOfBounds, BadMagic, TableMissing };
pub const ByteReader = struct {
    data: []const u8,
    pub fn u16At(self: ByteReader, off: usize) ParseError!u16 {
        if (off + 2 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(u16, self.data[off..][0..2], .big);
    }
    pub fn u32At(self: ByteReader, off: usize) ParseError!u32 {
        if (off + 4 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(u32, self.data[off..][0..4], .big);
    }
};

// The file opens with an offset table: a 4-byte version, then the table count, then
// three fields (searchRange/entrySelector/rangeShift) we can ignore.
pub const OffsetTable = struct { sfnt_version: u32, num_tables: u16 };

pub fn parseOffsetTable(r: ByteReader) ParseError!OffsetTable {
    const ver = try r.u32At(0);
    // 0x00010000 = TrueType outlines; 0x4F54544F = 'OTTO' = CFF/OpenType outlines.
    if (ver != 0x00010000 and ver != 0x4F54544F) return error.BadMagic;
    return .{ .sfnt_version = ver, .num_tables = try r.u16At(4) };
}

// Each of num_tables directory records is 16 bytes: a 4-byte tag, a checksum, the
// table's offset from the start of the file, and its length. We scan for a tag.
pub const TableRecord = struct { offset: u32, length: u32 };

pub fn findTable(r: ByteReader, num_tables: u16, tag: []const u8) ParseError!TableRecord {
    var i: usize = 0;
    while (i < num_tables) : (i += 1) {
        const base = 12 + i * 16;
        if (base + 16 > r.data.len) return error.OutOfBounds;
        if (std.mem.eql(u8, r.data[base..][0..4], tag)) {
            return .{ .offset = try r.u32At(base + 8), .length = try r.u32At(base + 12) };
        }
    }
    return error.TableMissing;
}

test "parse the offset table and find a table by its four-byte tag" {
    var buf: [28]u8 = undefined; // 12-byte offset table + one 16-byte record
    @memset(&buf, 0);
    std.mem.writeInt(u32, buf[0..4], 0x00010000, .big);
    std.mem.writeInt(u16, buf[4..6], 1, .big); // one table
    @memcpy(buf[12..16], "glyf");
    std.mem.writeInt(u32, buf[20..24], 100, .big); // record offset field (base+8)
    std.mem.writeInt(u32, buf[24..28], 42, .big); // record length field (base+12)

    const r = ByteReader{ .data = &buf };
    const ot = try parseOffsetTable(r);
    try std.testing.expectEqual(@as(u16, 1), ot.num_tables);
    const rec = try findTable(r, ot.num_tables, "glyf");
    try std.testing.expectEqual(@as(u32, 100), rec.offset);
    try std.testing.expectEqual(@as(u32, 42), rec.length);
    try std.testing.expectError(error.TableMissing, findTable(r, ot.num_tables, "cmap"));
}

A linear scan looks lazy next to the binary-search fields the format provides, but a font has a few dozen tables and a clear loop beats a clever bisection that saves a microsecond once. The error.TableMissing matters: a font with no glyf is not a TrueType outline font, and the caller needs that as an error, not a wrong answer.

head, maxp and turning a glyph id into bytes

Two tables get us from "a font" to "the bytes of a glyph". head holds unitsPerEm (the coordinate grid, 1000 or 2048) at byte 18, and indexToLocFormat at byte 50. maxp gives numGlyphs at byte 4. And loca is the index: numGlyphs + 1 offsets into glyf, where glyph i occupies loca[i] up to loca[i+1]. Equal neighbours mean an empty glyph -- exactly how a space is stored:

const std = @import("std");
pub const ParseError = error{ OutOfBounds, BadLocaFormat };
pub const ByteReader = struct {
    data: []const u8,
    pub fn u16At(self: ByteReader, off: usize) ParseError!u16 {
        if (off + 2 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(u16, self.data[off..][0..2], .big);
    }
    pub fn u32At(self: ByteReader, off: usize) ParseError!u32 {
        if (off + 4 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(u32, self.data[off..][0..4], .big);
    }
    pub fn i16At(self: ByteReader, off: usize) ParseError!i16 {
        if (off + 2 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(i16, self.data[off..][0..2], .big);
    }
};

// The 'head' table holds font-wide fields. Two matter for reading outlines:
// unitsPerEm (the coordinate grid a glyph lives on) at byte 18, and indexToLocFormat
// at byte 50 -- 0 means the loca table uses short (u16) offsets, 1 means long (u32).
pub const Head = struct { units_per_em: u16, loc_format: i16 };

pub fn parseHead(r: ByteReader, head_off: usize) ParseError!Head {
    return .{
        .units_per_em = try r.u16At(head_off + 18),
        .loc_format = try r.i16At(head_off + 50),
    };
}

// The 'loca' table is an array of numGlyphs+1 offsets into 'glyf'. Glyph i lives in
// bytes [loca[i], loca[i+1]); equal offsets mean an empty glyph (a space). Short
// format stores offset/2 as u16 (so we double it), long stores the u32 directly.
pub const GlyfRange = struct { start: u32, end: u32 };

pub fn glyfRange(r: ByteReader, loca_off: usize, loc_format: i16, id: u16) ParseError!GlyfRange {
    if (loc_format == 0) {
        const a = try r.u16At(loca_off + @as(usize, id) * 2);
        const b = try r.u16At(loca_off + (@as(usize, id) + 1) * 2);
        return .{ .start = @as(u32, a) * 2, .end = @as(u32, b) * 2 };
    } else if (loc_format == 1) {
        const a = try r.u32At(loca_off + @as(usize, id) * 4);
        const b = try r.u32At(loca_off + (@as(usize, id) + 1) * 4);
        return .{ .start = a, .end = b };
    } else return error.BadLocaFormat;
}

test "read head fields and resolve a glyph's byte range from a short-format loca" {
    var head: [54]u8 = undefined;
    @memset(&head, 0);
    std.mem.writeInt(u16, head[18..20], 2048, .big); // unitsPerEm
    std.mem.writeInt(i16, head[50..52], 0, .big); // short loca format
    const hr = ByteReader{ .data = &head };
    const h = try parseHead(hr, 0);
    try std.testing.expectEqual(@as(u16, 2048), h.units_per_em);
    try std.testing.expectEqual(@as(i16, 0), h.loc_format);

    // loca (short): entries 0, 0, 5 -> glyph 0 empty [0,0), glyph 1 is [0,10)
    var loca: [6]u8 = undefined;
    std.mem.writeInt(u16, loca[0..2], 0, .big);
    std.mem.writeInt(u16, loca[2..4], 0, .big);
    std.mem.writeInt(u16, loca[4..6], 5, .big);
    const lr = ByteReader{ .data = &loca };
    const g0 = try glyfRange(lr, 0, h.loc_format, 0);
    const g1 = try glyfRange(lr, 0, h.loc_format, 1);
    try std.testing.expectEqual(@as(u32, 0), g0.end - g0.start); // empty
    try std.testing.expectEqual(@as(u32, 0), g1.start);
    try std.testing.expectEqual(@as(u32, 10), g1.end); // 5 * 2
}

The short-format doubling is the gotcha: each loca entry is the real offset divided by two, so you multiply back (glyph data is always word-aligned). And the empty-glyph convention is elegant -- a space needs no outline, its range is zero bytes, and any code asking for it gets "nothing here" for free.

A glyph header, and tying it all together

Inside glyf each glyph opens with a header: a signed contour count and a bounding box. The sign is load-bearing -- a negative count means a composite glyph built from other glyphs (an accented 'e' is 'e' plus an acute), while zero-or-more is a simple glyph. We do not decode the points today, but reading the header proves the pipeline. Here is the payoff -- a TrueTypeFont that parses the directory once and resolves any glyph id through loca into glyf:

const std = @import("std");
pub const ParseError = error{ OutOfBounds, BadMagic, TableMissing, BadLocaFormat };

pub const ByteReader = struct {
    data: []const u8,
    pub fn u16At(self: ByteReader, off: usize) ParseError!u16 {
        if (off + 2 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(u16, self.data[off..][0..2], .big);
    }
    pub fn u32At(self: ByteReader, off: usize) ParseError!u32 {
        if (off + 4 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(u32, self.data[off..][0..4], .big);
    }
    pub fn i16At(self: ByteReader, off: usize) ParseError!i16 {
        if (off + 2 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(i16, self.data[off..][0..2], .big);
    }
};

fn parseOffsetTableCount(r: ByteReader) ParseError!u16 {
    const ver = try r.u32At(0);
    if (ver != 0x00010000 and ver != 0x4F54544F) return error.BadMagic;
    return r.u16At(4);
}

const TableRecord = struct { offset: u32, length: u32 };
fn findTable(r: ByteReader, num_tables: u16, tag: []const u8) ParseError!TableRecord {
    var i: usize = 0;
    while (i < num_tables) : (i += 1) {
        const base = 12 + i * 16;
        if (base + 16 > r.data.len) return error.OutOfBounds;
        if (std.mem.eql(u8, r.data[base..][0..4], tag))
            return .{ .offset = try r.u32At(base + 8), .length = try r.u32At(base + 12) };
    }
    return error.TableMissing;
}

// A glyph in 'glyf' opens with a header: the contour count (negative means it is a
// composite made of other glyphs) and the bounding box in font units.
pub const GlyphHeader = struct {
    number_of_contours: i16,
    x_min: i16,
    y_min: i16,
    x_max: i16,
    y_max: i16,
    pub fn isComposite(self: GlyphHeader) bool {
        return self.number_of_contours < 0;
    }
};

// Tie the tables together. parse() walks the directory once and caches what we need;
// glyphHeader() then resolves any glyph id through loca into glyf, returning null for
// an empty glyph (a space) and an error for an id past the font's glyph count.
pub const TrueTypeFont = struct {
    data: []const u8,
    units_per_em: u16,
    num_glyphs: u16,
    loc_format: i16,
    loca_off: usize,
    glyf_off: usize,

    pub fn parse(data: []const u8) ParseError!TrueTypeFont {
        const r = ByteReader{ .data = data };
        const num_tables = try parseOffsetTableCount(r);
        const head_rec = try findTable(r, num_tables, "head");
        const maxp_rec = try findTable(r, num_tables, "maxp");
        const loca_rec = try findTable(r, num_tables, "loca");
        const glyf_rec = try findTable(r, num_tables, "glyf");
        return .{
            .data = data,
            .units_per_em = try r.u16At(head_rec.offset + 18),
            .num_glyphs = try r.u16At(maxp_rec.offset + 4),
            .loc_format = try r.i16At(head_rec.offset + 50),
            .loca_off = loca_rec.offset,
            .glyf_off = glyf_rec.offset,
        };
    }

    fn glyfRange(self: TrueTypeFont, id: u16) ParseError!struct { start: u32, end: u32 } {
        const r = ByteReader{ .data = self.data };
        if (self.loc_format == 0) {
            const a = try r.u16At(self.loca_off + @as(usize, id) * 2);
            const b = try r.u16At(self.loca_off + (@as(usize, id) + 1) * 2);
            return .{ .start = @as(u32, a) * 2, .end = @as(u32, b) * 2 };
        }
        const a = try r.u32At(self.loca_off + @as(usize, id) * 4);
        const b = try r.u32At(self.loca_off + (@as(usize, id) + 1) * 4);
        return .{ .start = a, .end = b };
    }

    pub fn glyphHeader(self: TrueTypeFont, id: u16) ParseError!?GlyphHeader {
        if (id >= self.num_glyphs) return error.OutOfBounds;
        const range = try self.glyfRange(id);
        if (range.end <= range.start) return null; // empty glyph, e.g. a space
        const r = ByteReader{ .data = self.data };
        const base = self.glyf_off + range.start;
        return GlyphHeader{
            .number_of_contours = try r.i16At(base + 0),
            .x_min = try r.i16At(base + 2),
            .y_min = try r.i16At(base + 4),
            .x_max = try r.i16At(base + 6),
            .y_max = try r.i16At(base + 8),
        };
    }
};

fn buildTinyFont(buf: *[178]u8) void {
    @memset(buf, 0);
    std.mem.writeInt(u32, buf[0..4], 0x00010000, .big);
    std.mem.writeInt(u16, buf[4..6], 4, .big); // four tables
    const Rec = struct { tag: []const u8, off: u32, len: u32 };
    const recs = [_]Rec{
        .{ .tag = "head", .off = 76, .len = 54 },
        .{ .tag = "maxp", .off = 130, .len = 32 },
        .{ .tag = "loca", .off = 162, .len = 6 },
        .{ .tag = "glyf", .off = 168, .len = 10 },
    };
    for (recs, 0..) |rec, i| {
        const b = 12 + i * 16;
        @memcpy(buf[b .. b + 4], rec.tag);
        std.mem.writeInt(u32, buf[b + 8 ..][0..4], rec.off, .big);
        std.mem.writeInt(u32, buf[b + 12 ..][0..4], rec.len, .big);
    }
    std.mem.writeInt(u16, buf[76 + 18 ..][0..2], 1000, .big); // unitsPerEm
    std.mem.writeInt(i16, buf[76 + 50 ..][0..2], 0, .big); // short loca
    std.mem.writeInt(u16, buf[130 + 4 ..][0..2], 2, .big); // numGlyphs = 2
    std.mem.writeInt(u16, buf[162..][0..2], 0, .big); // loca[0]
    std.mem.writeInt(u16, buf[164..][0..2], 0, .big); // loca[1] (glyph 0 empty)
    std.mem.writeInt(u16, buf[166..][0..2], 5, .big); // loca[2] -> glyph 1 ends at 10
    std.mem.writeInt(i16, buf[168..][0..2], 1, .big); // glyph 1: 1 contour
    std.mem.writeInt(i16, buf[170..][0..2], 0, .big); // xMin
    std.mem.writeInt(i16, buf[172..][0..2], 0, .big); // yMin
    std.mem.writeInt(i16, buf[174..][0..2], 700, .big); // xMax
    std.mem.writeInt(i16, buf[176..][0..2], 700, .big); // yMax
}

test "parse a whole (tiny) font and read a glyph header, empty glyph comes back null" {
    var buf: [178]u8 = undefined;
    buildTinyFont(&buf);
    const font = try TrueTypeFont.parse(&buf);
    try std.testing.expectEqual(@as(u16, 1000), font.units_per_em);
    try std.testing.expectEqual(@as(u16, 2), font.num_glyphs);

    try std.testing.expect((try font.glyphHeader(0)) == null); // empty glyph
    const g1 = (try font.glyphHeader(1)).?;
    try std.testing.expectEqual(@as(i16, 1), g1.number_of_contours);
    try std.testing.expect(!g1.isComposite());
    try std.testing.expectEqual(@as(i16, 700), g1.x_max);
    try std.testing.expectError(error.OutOfBounds, font.glyphHeader(2)); // past numGlyphs
}

Look what the type system bought us: glyphHeader returns ParseError!?GlyphHeader -- either an error (id out of range, truncated file), or null (a valid empty glyph), or a real header. Three outcomes, each forced on the caller, none silently confused. I build the whole font from a hand-made byte array in the test, which is the honest way to test a parser: you know every byte in, so you know exactly what must come out.

Mapping a character to a glyph: cmap

One table remains before real text: cmap, which answers "what glyph draws 'A'?". It carries several subtables in different encodings. The simplest, format 0, is a flat 256-entry byte array -- a single-byte character indexes straight into it. Real Unicode fonts use format 4 (segmented 16-bit ranges), which we will tackle later; format 0 shows the shape and is fully testable:

const std = @import("std");
pub const ParseError = error{ OutOfBounds, UnsupportedCmap };
pub const ByteReader = struct {
    data: []const u8,
    pub fn u16At(self: ByteReader, off: usize) ParseError!u16 {
        if (off + 2 > self.data.len) return error.OutOfBounds;
        return std.mem.readInt(u16, self.data[off..][0..2], .big);
    }
};

// The 'cmap' table maps characters to glyph ids and holds several subtables in
// different encodings. The simplest, format 0, is a flat 256-entry byte array that
// maps a single-byte character straight to a glyph id. Real Unicode fonts use format
// 4 (segmented ranges of 16-bit code points), which we describe but leave for later.
pub fn glyphIdFormat0(r: ByteReader, subtable_off: usize, ch: u8) ParseError!u8 {
    const format = try r.u16At(subtable_off);
    if (format != 0) return error.UnsupportedCmap;
    // layout: format(2) length(2) language(2) then 256 bytes of glyph ids
    const table = subtable_off + 6;
    if (table + 256 > r.data.len) return error.OutOfBounds;
    return r.data[table + ch];
}

test "a format-0 cmap maps a byte straight to its glyph id, unmapped falls to .notdef" {
    var buf: [6 + 256]u8 = undefined;
    @memset(&buf, 0);
    std.mem.writeInt(u16, buf[0..2], 0, .big); // format 0
    std.mem.writeInt(u16, buf[2..4], 6 + 256, .big); // length
    buf[6 + @as(usize, 'A')] = 36; // 'A' -> glyph 36
    buf[6 + @as(usize, 'B')] = 37; // 'B' -> glyph 37
    const r = ByteReader{ .data = &buf };
    try std.testing.expectEqual(@as(u8, 36), try glyphIdFormat0(r, 0, 'A'));
    try std.testing.expectEqual(@as(u8, 37), try glyphIdFormat0(r, 0, 'B'));
    try std.testing.expectEqual(@as(u8, 0), try glyphIdFormat0(r, 0, 'Z')); // unmapped -> 0
}

An unmapped character landing on glyph 0 is not a bug, it is the convention: glyph 0 is always .notdef, the little box you see for a missing character. So "not found" and "found glyph 0" are deliberately the same answer. Format 4 is more work, but the interface is identical: bytes in, glyph id out.

The same reader in C, Rust and Go

Reading a big-endian integer and checking you did not run off the end is the same job everywhere; what differs is who guarantees the bounds. C does the read by hand and the bounds check is yours to remember:

// C: big-endian read by hand; YOU must bounds-check before calling
#include 
uint32_t u32_at(const uint8_t *data, size_t len, size_t off, int *ok) {
    if (off + 4 > len) { *ok = 0; return 0; }   // forget this line and you read garbage
    return ((uint32_t)data[off] << 24) | ((uint32_t)data[off+1] << 16)
         | ((uint32_t)data[off+2] << 8) | (uint32_t)data[off+3];
}

Rust folds the bounds check into a slice conversion and names the endianness:

// Rust: the slice conversion is the bounds check; endianness is explicit
fn u32_at(data: &[u8], off: usize) -> Option<u32> {
    let bytes: [u8; 4] = data.get(off..off + 4)?.try_into().ok()?;
    Some(u32::from_be_bytes(bytes))
}

Go bounds-checks every slice access at runtime and ships a big-endian helper:

// Go: encoding/binary has the read; the runtime bounds-checks the slice
import "encoding/binary"
func u32At(data []byte, off int) (uint32, bool) {
    if off+4 > len(data) { return 0, false }
    return binary.BigEndian.Uint32(data[off : off+4]), true
}

C is fastest and least forgiving. Rust folds the check into the type conversion. Go pays a runtime check. Zig proves the read width at compile time and returns a typed error for the bounds check. The algorithm is identical everywhere -- respect the byte order, never trust the length.

Exercises

  1. Read hhea and hmtx for advance widths. Parse numberOfHMetrics from hhea (byte 34), then read the advanceWidth (u16) for a glyph from hmtx, remembering glyphs past numberOfHMetrics share the last entry's advance. Test against a hand-built hmtx.

  2. Composite glyph detection. Add a kind that reports simple, composite or empty for a glyph id, using the sign of the contour count and the empty-range check. Build a tiny font with one of each and assert the classification.

  3. A cmap subtable picker. The cmap table starts with a count then one record per subtable (platform id, encoding id, offset). Write a function that returns the offset of the best subtable (prefer Unicode), so the format-0 reader can be pointed at whatever a real file provides.

What we learned

  • A bitmap font is locked to one size; drawing type at any size means reading a font file's outlines, which starts with parsing the file's structure;
  • A TrueType file is an offset table plus a directory of named tables; you reach any table by finding its four-byte tag and jumping to its offset;
  • Every number is big-endian, so a bounds-checked reader is the foundation -- font files are untrusted input;
  • head gives unitsPerEm and the loca format, maxp the glyph count, and loca turns a glyph id into a byte range in glyf -- equal neighbours meaning an empty glyph;
  • A glyph header's signed contour count distinguishes simple from composite; ParseError!?GlyphHeader keeps error, empty and present as three outcomes the compiler forces you to separate;
  • cmap maps a character to a glyph id; format 0 is a flat 256-byte table, format 4 is segmented Unicode, and glyph 0 is the deliberate .notdef fallback;
  • The job is identical in C, Rust and Go, and Zig proves the read width at compile time while returning a typed error.

We now hold the map of a font file, down to a glyph's outline header. What we have not done is read the outline itself: the on-curve and off-curve points, the quadratic curves, and the rasterizer that fills them at any size. Before we get there we need something we have hand-waved since episode 156: how colors actually mix when one shape is drawn, half-transparent, over another. Anti-aliased type is nothing but careful color mixing along a curve's edge, so that is where we go next. Plenty still to build.

Thanks for reading, and tot ziens! ;-)

scipio@scipio

stem
stemsocial
steemstem
zig
programming
via hiveblog
This post was published via hiveblog. Ecency is not the originator or editor of this content and displays it for discovery purposes only.
16
Learn Zig Series (#164) - TrueType Parsing | Ecency