Learn Rust Series (#60) - Mini Project: A Fully Tested, Documented, Published-Ready CSV Toolkit Crate

scipio(71)

Published in

StemSocial

Words

3327

Reading

15 min

Listen

Play

Learn Rust Series (#60) - Mini Project: A Fully Tested, Documented, Published-Ready CSV Toolkit Crate

What will I learn

You will pull together everything from Phase 4 into one small but complete library crate;
how to design a clean error type, a line parser that respects quoted fields, and a Table abstraction;
how to select a column by name and write records back out, returning Result at every fallible step;
how to test the crate with unit tests and doctests so the examples in your docs never rot;
what "publish-ready" really means -- docs, tests, a README, and a Cargo.toml fit for crates.io.

Requirements

A working modern computer running macOS, Windows or Ubuntu, with Cargo;
An installed Rust toolchain (via rustup, from rustup.rs);
The previous fifty-nine episodes, especially error handling, testing, and idiomatic Rust;
The ambition to learn systems programming from the ground up.

Difficulty

Intermediate

Curriculum (of the `Learn Rust Series`):

Learn Rust Series (#60) - Mini Project: A Fully Tested, Documented, Published-Ready CSV Toolkit Crate

This is the Phase 4 capstone, and it ties the whole phase together. Over the last stretch of episodes we learned how a crate can test itself, benchmark itself, split into a workspace, compile conditionally, generate parts of itself at build time, and hold itself to a consistent style. What we have never done is put all of that discipline into one small, real, end-to-end thing you could actually hand to another person. So that is exactly what we will do now: build a little CSV toolkit crate from scratch -- no external CSV library, just std -- with a proper error type, a parser that handles quoted fields, a Table you can query by column name, and a serializer to write data back out. Along the way we test it with unit tests and doctests, and finish by talking about what it takes to make a crate genuinely publish-ready. It is precisely the kind of focused, well-tested library that Rust makes a pleasure to write ;-)

CSV is a lovely target for this, by the way, because everyone thinks it is trivial ("just split on commas!") and then reality intrudes the moment a field contains a comma of its own. That gap between the naive version and the correct version is where most of the interesting design lives, so it teaches more than its size suggests. Let us first clear last episode's homework, and then build the crate step by step.

Solutions to Episode 59 Exercises

Episode 59 was clippy, rustfmt, and idiomatic Rust. All three exercises were about spotting a clumsy shape and reaching for the fluent one instead.

Exercise 1 asked you to take a for i in 0..v.len() loop that builds a new Vec of each element squared, and rewrite it as an iterator chain. The needless_range_loop disappears the moment the index does:

fn squares(v: &[i32]) -> Vec<i32> {
    // was: let mut out = Vec::new(); for i in 0..v.len() { out.push(v[i] * v[i]); } out
    v.iter().map(|x| x * x).collect()
}

fn main() {
    println!("{:?}", squares(&[1, 2, 3, 4])); // [1, 4, 9, 16]
}

The iterator form says what it means -- "each element, squared, collected" -- with no manual counter and no per-access bounds check. Clippy nudges you here precisely because the index i was pure ceremony.

Exercise 2 wanted a match on a three-variant enum that returns true for one variant and false for the rest, replaced with a single matches! call:

#[derive(Debug)]
enum State { Idle, Running, Done }

fn is_running(s: &State) -> bool {
    // was: match s { State::Running => true, _ => false }
    matches!(s, State::Running)
}

fn main() {
    println!("{}", is_running(&State::Running)); // true
    println!("{}", is_running(&State::Idle));    // false
    println!("{}", is_running(&State::Done));    // false
}

matches! collapses the whole "map every arm to a bool" pattern into one expression, and it is exactly what clippy's match_like_matches_macro suggests. Behaviour is identical for every variant -- only the noise is gone.

Exercise 3 asked you to take a function that accepts &String and only reads it, change the parameter to &str, then call the new version with both a &String and a string literal to prove &str accepts both:

fn shout(s: &str) -> String {
    format!("{}!", s.to_uppercase())
}

fn main() {
    let owned = String::from("hive");
    println!("{}", shout(&owned));   // &String coerces to &str -- for free
    println!("{}", shout("rust"));   // a string literal is already &str
}

Taking &str demands strictly less of the caller: through the deref coercion we met back in the Deref episode, a &String slides into a &str parameter at the call site with zero cost, while a &String parameter would have rejected the literal. The ptr_arg lint is the one that flags the &String and points you here. Right, homework cleared. Now, the toolkit.

Step 1: the error type

A library returns precise, matchable errors rather than throwing strings around, so we start exactly where episode 48 told us to: an enum covering the ways CSV handling can fail, with the Display and Error impls that make it a first-class citizen of the std::error::Error world. Two failure modes are enough for our toolkit -- a row whose field count does not match the header, and a request for a column that does not exist:

use std::fmt;

#[derive(Debug, PartialEq)]
pub enum CsvError {
    RaggedRow { row: usize, expected: usize, found: usize },
    UnknownColumn(String),
}

impl fmt::Display for CsvError {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        match self {
            CsvError::RaggedRow { row, expected, found } =>
                write!(f, "row {row} has {found} fields, expected {expected}"),
            CsvError::UnknownColumn(name) => write!(f, "no column named '{name}'"),
        }
    }
}

impl std::error::Error for CsvError {}

The #[derive(PartialEq)] is not decoration -- it lets our tests write assert_eq!(result, Err(CsvError::UnknownColumn("email".into()))) and compare errors by value, which is enormously convenient. Deriving Debug gives us the {:?} formatting that assert_eq! needs on failure. And implementing Display plus the empty impl std::error::Error means a caller can slot our error straight into a Box<dyn Error> or an anyhow::Result without any friction. That is the whole point of the trait: it is the shared vocabulary every Rust error speaks.

Step 2: parsing a line, the naive way

The temptation is to write the clever parser first. Resist it. The naive split(',') version is worth writing on its own, because it is genuinely fine for simple, well-behaved data, and because seeing its one flaw makes the quoted version's job obvious:

fn split_simple(line: &str) -> Vec<String> {
    line.split(',').map(|s| s.trim().to_string()).collect()
}

fn main() {
    let fields = split_simple("alice, 30, engineer");
    println!("{fields:?}"); // ["alice", "30", "engineer"]
}

Clean, readable, and correct -- right up until a field wants to contain a comma. split_simple("alice,\"Portland, OR\",engineer") would hand you back four fields in stead of three, splitting the city and state apart. That single failure is the entire reason real CSV has a quoting rule, so let us honour it.

Step 3: respecting quoted fields

Real CSV lets a field be wrapped in double quotes so it can contain commas. Handling that needs a tiny state machine that tracks whether we are currently inside a quoted field, and only treats a comma as a separator when we are outside one:

fn split_quoted(line: &str) -> Vec<String> {
    let mut fields = Vec::new();
    let mut current = String::new();
    let mut in_quotes = false;
    for ch in line.chars() {
        match ch {
            '"' => in_quotes = !in_quotes,
            ',' if !in_quotes => fields.push(std::mem::take(&mut current)),
            other => current.push(other),
        }
    }
    fields.push(current); // the last field has no trailing comma to flush it
    fields
}

fn main() {
    let fields = split_quoted(r#"alice,"Portland, OR",engineer"#);
    println!("{fields:?}"); // ["alice", "Portland, OR", "engineer"]
}

Notice the std::mem::take from episode 43. When we hit a separating comma we want to push the finished field and leave a fresh empty String behind to accumulate the next one, and we want to do it without cloning. std::mem::take(&mut current) does exactly that: it moves the built-up String out, hands it to push, and replaces current with String::default() (an empty string) in a single move. This is the kind of small, allocation-free trick that separates code that works from code that works and does not waste heap traffic. The match guard ',' if !in_quotes is also a nice callback to the pattern-matching episode -- the guard is what lets a comma mean two different things depending on state.

Step 4: a Table with headers, checked

Now the central abstraction. A Table is a header row plus a set of data rows. Rather than a loose parse that trusts its input, we build a checked constructor that validates every row against the header width and reports a RaggedRow error the moment something does not line up. This is where the first arm of our error enum earns its keep:

pub struct Table {
    pub headers: Vec<String>,
    pub rows: Vec<Vec<String>>,
}

impl Table {
    /// Parse CSV text into a Table, validating that every row has exactly as
    /// many fields as there are headers.
    pub fn parse(text: &str) -> Result<Table, CsvError> {
        let mut lines = text.lines();
        let headers: Vec<String> = lines
            .next().unwrap_or("")
            .split(',').map(|s| s.trim().to_string()).collect();

        let width = headers.len();
        let mut rows = Vec::new();
        for (i, line) in lines.enumerate() {
            let row: Vec<String> = line.split(',').map(|s| s.trim().to_string()).collect();
            if row.len() != width {
                return Err(CsvError::RaggedRow { row: i, expected: width, found: row.len() });
            }
            rows.push(row);
        }
        Ok(Table { headers, rows })
    }

    pub fn width(&self) -> usize {
        self.headers.len()
    }
}

fn main() {
    let good = Table::parse("name,age\nalice,30\nbob,25").unwrap();
    println!("{} columns, {} rows", good.width(), good.rows.len()); // 2 columns, 2 rows

    let bad = Table::parse("name,age\nalice"); // one field, expected two
    println!("{:?}", bad.err()); // Some(RaggedRow { row: 0, expected: 2, found: 1 })
}

The type signature Result<Table, CsvError> is doing real communication here: it tells every caller, before they read a line of the body, that construction can fail and that they must decide what to do about it. That is the honesty the whole of Phase 4 has been about. A malformed CSV file does not silently produce a lopsided Table full of surprises later -- it fails loudly and immediately, with a row number and the exact counts, so the caller can print something a human can act on.

Step 5: selecting a column by name

Here is the query users actually want: "give me the age column". This is a fallible lookup, because the name might not exist, so it returns Result and reports UnknownColumn on a miss. It leans on the combinators from episodes 46 and 47 to stay a single clean expression:

impl Table {
    pub fn column(&self, name: &str) -> Result<Vec<&str>, CsvError> {
        let idx = self
            .headers
            .iter()
            .position(|h| h == name)
            .ok_or_else(|| CsvError::UnknownColumn(name.to_string()))?;
        Ok(self.rows.iter().map(|r| r[idx].as_str()).collect())
    }
}

fn main() {
    let t = Table::parse("name,age\nalice,30\nbob,25").unwrap();
    println!("{:?}", t.column("age"));            // Ok(["30", "25"])
    println!("{}", t.column("email").is_err());   // true
}

Two idioms are worth pausing on. First, position returns an Option<usize> -- Some(i) if the header is found, None otherwise -- and ok_or_else turns that None into our UnknownColumn error, at which point the ? short-circuits and returns it. Using ok_or_else rather than ok_or means we only allocate the name.to_string() on the actual error path, not on every successful lookup; a tiny thing, but exactly the kind of tiny thing clippy taught us to notice last episode. Second, we return Vec<&str> -- borrowed slices into the table's own strings -- rather than cloning every value into fresh Strings. The lifetime elision from episode 44 ties those borrows to &self, so the compiler guarantees the returned slices cannot outlive the table they point into. Zero copies, full safety. Bam.

Step 6: writing CSV back out

A toolkit that only reads is half a toolkit. Serialization is the mirror image of parsing: join the fields of each row with commas, and join the rows with newlines. We put the header row first so the output round-trips back through Table::parse:

pub fn to_csv(headers: &[String], rows: &[Vec<String>]) -> String {
    let mut out = String::new();
    out.push_str(&headers.join(","));
    out.push('\n');
    for row in rows {
        out.push_str(&row.join(","));
        out.push('\n');
    }
    out
}

fn main() {
    let headers = vec!["name".to_string(), "age".to_string()];
    let rows = vec![
        vec!["alice".to_string(), "30".to_string()],
        vec!["bob".to_string(), "25".to_string()],
    ];
    print!("{}", to_csv(&headers, &rows));
    // name,age
    // alice,30
    // bob,25
}

The join(",") method on a slice of strings is std doing the boring work for us: it inserts the separator between elements and not after the last one, which is subtly different from pushing a comma after every field. This naive writer does not yet quote fields that contain commas -- that is deliberately left as one of today's exercises, because doing it correctly is the exact inverse of the parsing state machine and a great way to prove you understood Step 3.

Step 7: documentation that cannot lie

Now we make the crate's public surface teach itself. Rust's killer feature for library authors is the doctest: a code example inside a /// doc comment is compiled and run by cargo test, so an example that drifts out of date fails your build instead of quietly misleading your readers. We met these in episode 52; here is column with an example that doubles as a test:

impl Table {
    /// Return a column's values by header name.
    ///
    /// # Errors
    /// Returns [`CsvError::UnknownColumn`] if no header matches `name`.
    ///
    /// # Examples
    /// ```
    /// use csv_toolkit::Table;
    /// let t = Table::parse("name,age\nalice,30\nbob,25").unwrap();
    /// assert_eq!(t.column("age").unwrap(), vec!["30", "25"]);
    /// assert!(t.column("email").is_err());
    /// ```
    pub fn column(&self, name: &str) -> Result<Vec<&str>, CsvError> {
        let idx = self
            .headers
            .iter()
            .position(|h| h == name)
            .ok_or_else(|| CsvError::UnknownColumn(name.to_string()))?;
        Ok(self.rows.iter().map(|r| r[idx].as_str()).collect())
    }
}

Read that example the way rustdoc does: it is real code, run against the real published API, using the crate exactly as an outside user would (use csv_toolkit::Table;). If a future refactor renames column or changes its return type, this doctest goes red. That is the quiet superpower -- your documentation and your test suite become the same artifact, so they can never disagree. The # Errors and # Examples headings are also the conventional sections rustdoc renders specially, so following them makes your docs look like the standard library's.

Step 8: testing the tricky parts

Doctests cover the public examples; unit tests pin down the internal corners, above all the quoted-comma case that motivated the whole state machine. Following the convention from episode 52, the tests live in a #[cfg(test)] module right beside the code they exercise:

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn handles_quoted_commas() {
        assert_eq!(split_quoted(r#"a,"b,c",d"#), vec!["a", "b,c", "d"]);
    }

    #[test]
    fn plain_line_is_unaffected() {
        assert_eq!(split_quoted("x,y,z"), vec!["x", "y", "z"]);
    }

    #[test]
    fn ragged_rows_are_rejected() {
        let err = Table::parse("a,b\n1").unwrap_err();
        assert_eq!(err, CsvError::RaggedRow { row: 0, expected: 2, found: 1 });
    }

    #[test]
    fn missing_column_errors() {
        let t = Table::parse("name,age\nalice,30").unwrap();
        assert_eq!(t.column("nope"), Err(CsvError::UnknownColumn("nope".into())));
    }
}

Every one of these tests is a claim about behaviour that a future change could break. handles_quoted_commas guards the one line of parsing logic that is easy to get wrong; ragged_rows_are_rejected and missing_column_errors prove our error paths actually fire, and -- thanks to that #[derive(PartialEq)] on CsvError -- they assert on the exact error value, not just "some error happened". cargo test runs the unit tests, the integration tests, and the doctests in one shot, so a green run means the code, the internal invariants, and the public examples are all consistent. That is the trustworthiness bar a crate should clear before anyone depends on it.

Step 9: what "publish-ready" actually means

Compiling and passing tests is necessary but not sufficient. Before cargo publish will (or should) push your crate to crates.io, it wants metadata a stranger can evaluate. At minimum that is a Cargo.toml with a description, a license, and a repository, plus a README.md and doc comments on every public item:

# Cargo.toml
[package]
name = "csv-toolkit"
version = "0.1.0"
edition = "2021"
description = "A tiny std-only CSV parser and writer"
license = "MIT OR Apache-2.0"
repository = "https://example.com/csv-toolkit"
readme = "README.md"
keywords = ["csv", "parser", "no-dependencies"]
categories = ["parsing"]

[dependencies]
# none -- this whole toolkit is std-only, which is a selling point

The license field using the SPDX expression MIT OR Apache-2.0 is the near-universal Rust convention, giving downstream users their choice of the two permissive licenses the language itself uses. An empty [dependencies] section is genuinely a feature to advertise: a crate with zero dependencies cannot drag a supply chain of transitive packages into someone else's build. And once the metadata is in place, the finalize ritual is a short checklist: cargo fmt --check, cargo clippy -- -D warnings, cargo test (unit, integration, and doctests all green), cargo doc --open to eyeball the rendered docs, and finally cargo publish --dry-run to see exactly what would be uploaded before you commit to the real thing. The --dry-run is your last, cheap safety net -- use it every single time.

Assembling the crate

Pulled together, the whole thing is one modest src/lib.rs -- the module layout we studied in episode 9, now carrying real weight:

//! # csv-toolkit
//!
//! A tiny, std-only CSV parser and writer: parse text into a validated
//! [`Table`], select columns by name, and serialize records back out.
//! Every fallible operation returns a [`Result`] with a precise [`CsvError`].

mod error;   // the CsvError enum + Display/Error impls
mod parse;   // split_simple, split_quoted, Table::parse
mod table;   // the Table type, column(), width()
mod write;   // to_csv

pub use error::CsvError;
pub use table::Table;
pub use write::to_csv;

The crate-level //! comment is the front page a visitor sees on docs.rs, and the tight pub use re-exports mean users write csv_toolkit::Table rather than digging through internal module paths -- the public API is small, flat, and honest, exactly as episode 28 argued a stable API should be. Everything private stays private; only the three things a user needs are exported.

Having said that, the deeper lesson of Phase 4 is not any single one of these mechanics -- it is the mindset they add up to. Fallible operations return Result instead of panicking or fibbing. Invariants (row width, column existence) are enforced at construction and covered by tests. The examples in the docs are compiled, so they cannot rot. The public surface is deliberately kept small. That collection of habits is the whole difference between "code that works on my machine" and "a crate other people can build a project on top of".

How Python and Go would frame this

A glance sideways sharpens the picture, as always. In Python you would almost never hand-roll this -- the standard library ships a battle-tested csv module, and reaching for it is the right call:

import csv, io

reader = csv.reader(io.StringIO('alice,"Portland, OR",engineer'))
print(next(reader))  # ['alice', 'Portland, OR', 'engineer']

That is Python's whole philosophy in miniature: batteries included, correctness handed to you, and the quoted-comma state machine already written and tested by someone else years ago. The cost is that the parser is a general-purpose black box you cannot easily specialise, and it produces lists of str with no compile-time guarantee about row shape.

Go sits closer to where we landed. Its encoding/csv is also in the standard library, but it returns explicit errors you are expected to check on the spot, which is the exact Result-shaped discipline we built by hand:

r := csv.NewReader(strings.NewReader("a,b\n1,2\n"))
records, err := r.ReadAll()
if err != nil { log.Fatal(err) } // errors are values, checked right here
fmt.Println(records) // [[a b] [1 2]]

The instructive contrast is that in Rust we did not need a library at all: std, the type system, and a dozen lines of a hand-written state machine gave us a parser that is safe, allocation-conscious, and fully our own -- and the Result return types make skipping the error check a compile error rather than a code-review nag. That "you can build it yourself, safely, from small pieces" feeling is a lot of why systems programmers fall for the language.

Wrapping up

So here is the shape of it. We started from an honest error type, wrote the naive parser to expose its one flaw, upgraded to a quoted-field state machine that leans on std::mem::take to stay allocation-free, wrapped it all in a Table whose checked constructor rejects ragged input, added a borrowing column selector that returns errors through clean combinators, and closed the loop with a serializer. Then we made it trustworthy -- doctests that double as documentation, unit tests that assert on exact error values -- and shippable, with the Cargo.toml, README, licensing, and dry-run ritual that turn a folder of code into a crate someone can cargo add. That is Phase 4, complete: everything from error design through testing, benchmarking, and tooling, brought to bear on one small library end to end.

Phase 5 changes the subject entirely, and it is the one a lot of you have been waiting for: concurrency. We touched threads and channels all the way back in episode 13, but now we go properly deep -- and the very first question we have to answer is what actually makes it safe for a value to cross a thread boundary at all. Rust has a famously elegant answer to that, baked right into the type system through two quiet little traits, and understanding them is what unlocks everything else that follows. That is where we are headed next ;-)

Exercises

Add a row(&self, i: usize) -> Option<&[String]> accessor to Table that returns the i-th data row as a borrowed slice, or None when i is past the end. Prove it returns None rather than panicking on an out-of-range index.
Write a quote-aware serializer to_csv_quoted that wraps any field containing a comma in double quotes (so Portland, OR becomes "Portland, OR"), leaving comma-free fields untouched. Confirm its output round-trips back through split_quoted.
Extend split_quoted to treat a doubled quote ("") inside a quoted field as a single literal " character, so that "she said ""hi""" parses to the single field she said "hi". This is the real CSV escaping rule -- watch your state transitions carefully.