PhantomData<T> is and how it makes a type "use" a parameter it does not store;PhantomData<&'a ()> without holding a real reference;Learn Rust Series):We have quietly leaned on PhantomData a few times already -- in the typestate episode (29) and again in the units-of-measure mini project (30) -- always waving it through as "a field that makes the type parameter count". Today I want to pay off that IOU and explain what it really is, and the surprising little family it belongs to: zero-sized types, values that take up no memory whatsoever yet still carry information the compiler acts on. They are a distinctively Rust idea, and they are the reason so much of the type-level safety we have been building costs exactly nothing at runtime ;-)
The last several episodes were the awkward-but-important corners of ownership -- moves, interior mutability, the reference-counted trio, Cow, and last time Pin. This one is lighter on machinery and heavier on a single elegant idea, and it closes the loop on every "marker" trick we have used without fully explaining. But first, as always, let me clear last episode's homework.
Episode 37 was Pin and self-referential structs. There were three exercises, and here is full runnable code for each.
Exercise 1 asked you to create a Pin<Box<u64>> with Box::pin, read the value through it, mutate it via Pin::get_mut, and explain in a comment why get_mut is allowed here:
use std::pin::Pin;
fn main() {
let mut p: Pin<Box<u64>> = Box::pin(100);
println!("start: {}", *p); // 100
// get_mut hands back a plain &mut u64. That is only sound because u64 is
// Unpin: moving a u64 can never invalidate a self-reference (it has none),
// so tearing the pin off is harmless and needs no `unsafe`.
*Pin::get_mut(p.as_mut()) += 1;
println!("after: {}", *p); // 101
}
The key insight is that Pin::get_mut is defined only for Unpin targets. Because u64 is Unpin, the pin is inert -- it is protecting a value that never needed protecting -- so the compiler lets you recover a mutable reference safely. Try the same call on a !Unpin type and get_mut simply is not in scope.
Exercise 2 wanted you to add a second self-referential pointer to SelfRef, set it inside new, add a method that reads through it, and confirm both self-pointers agree with direct field access. I added an owned len field and pointed at it:
use std::pin::Pin;
use std::marker::PhantomPinned;
use std::ptr;
struct SelfRef {
data: String,
len: usize,
ptr_to_data: *const String, // points into our own `data`
ptr_to_len: *const usize, // points into our own `len`
_pin: PhantomPinned, // makes SelfRef !Unpin
}
impl SelfRef {
fn new(text: &str) -> Pin<Box<SelfRef>> {
let mut boxed = Box::pin(SelfRef {
data: String::from(text),
len: text.len(),
ptr_to_data: ptr::null(),
ptr_to_len: ptr::null(),
_pin: PhantomPinned,
});
let data_ptr: *const String = &boxed.data;
let len_ptr: *const usize = &boxed.len;
// SAFETY: we only write two internal pointers aimed at our own pinned
// fields; we never move the value out, so both addresses stay valid.
unsafe {
let inner = Pin::get_unchecked_mut(Pin::as_mut(&mut boxed));
inner.ptr_to_data = data_ptr;
inner.ptr_to_len = len_ptr;
}
boxed
}
fn via_data_ptr(&self) -> &str {
unsafe { &*self.ptr_to_data } // SAFETY: points into our pinned `data`
}
fn via_len_ptr(&self) -> usize {
unsafe { *self.ptr_to_len } // SAFETY: points into our pinned `len`
}
}
fn main() {
let s = SelfRef::new("hello");
println!("data direct {} | via ptr {}", s.data, s.via_data_ptr()); // hello | hello
println!("len direct {} | via ptr {}", s.len, s.via_len_ptr()); // 5 | 5
}
Two self-pointers, same discipline: the struct is pinned on the heap so its fields have stable addresses, and both raw pointers keep aiming at the right bytes for the whole life of the value. The PhantomPinned field is doing the heavy lifting -- it is what makes SelfRef non-Unpin, and without it the pin would guard nothing.
Exercise 3 asked you to write fn needs_unpin<T: Unpin>(_: T) {}, define a struct containing a PhantomPinned field, try to pass it, and note which bound failed:
use std::marker::PhantomPinned;
fn needs_unpin<T: Unpin>(_: T) {}
struct Fixed {
_pin: PhantomPinned,
}
fn main() {
needs_unpin(42i32); // fine: i32 is Unpin
// needs_unpin(Fixed { _pin: PhantomPinned });
// ^ would NOT compile. The failing bound is `Fixed: Unpin`. Fixed inherited
// !Unpin from its PhantomPinned field, so it cannot satisfy T: Unpin.
let _ = Fixed { _pin: PhantomPinned };
}
The compiler error names the bound directly -- something like "the trait bound Fixed: Unpin is not satisfied" -- and points at the PhantomPinned field as the reason. That single zero-sized field is enough to opt the whole struct out of an auto trait, which is a perfect segue into today's topic. Right, homework cleared -- now, phantoms.
A zero-sized type, or ZST for short, is a type whose values occupy exactly zero bytes. This sounds like a paradox -- a value that is nothing? -- but Rust has several of them, and you have been using the most famous one since episode 1 without a second thought: the unit type (). A field-less struct and an empty-braces struct are ZSTs too:
struct Unit;
struct Empty {}
fn main() {
println!("() = {}", std::mem::size_of::<()>()); // 0
println!("Unit = {}", std::mem::size_of::<Unit>()); // 0
println!("Empty = {}", std::mem::size_of::<Empty>()); // 0
}
Every one of these prints 0. There is genuinely no data to store, so the compiler stores none. This is not a rounding-down or an approximation -- a ZST is guaranteed to have size zero, and the compiler builds real optimisations on top of that guarantee.
The most striking demonstration is that even a huge array of ZSTs costs nothing. An array [T; N] normally takes N * size_of::<T>() bytes, so with size_of::<T>() equal to zero, the whole array collapses to zero regardless of how many elements you ask for:
fn main() {
let big: [(); 1_000_000] = [(); 1_000_000];
println!("len = {}", big.len()); // 1000000
println!("bytes = {}", std::mem::size_of_val(&big)); // 0
}
A million-element array that weighs nothing. The length is still tracked (it is part of the type), iteration still visits it a million times, but there are literally no bytes backing it. That is the whole flavour of a ZST: it exists at the level of types and counts, but not at the level of memory.
Because a ZST is free to create and store, it is the idiomatic way to build a set out of a map. If you want a collection that answers "is this key present?" and stores nothing per key, you use a map whose value type is ():
use std::collections::HashMap;
fn main() {
let mut set: HashMap<&str, ()> = HashMap::new();
set.insert("rust", ());
set.insert("rust", ()); // duplicate key: still one entry
set.insert("zig", ());
println!("contains rust? {}", set.contains_key("rust")); // true
println!("size {}", set.len()); // 2
}
This is not a party trick -- it is quite literally how the standard library implements HashSet. Under the hood, HashSet<T> is a thin wrapper around HashMap<T, ()>, and the () values take up no space at all, so a set is exactly as cheap in memory as a map that only stored keys would be. The compiler knows the value slots are zero-sized and lays out the table without them. You get the ergonomics of a dedicated set type and pay nothing extra for the "value" half of the map.
Now to the star of the episode. Suppose you want a generic struct Wrapper<T>, parameterized by some T, but you do not actually want to store a T. Maybe T is only there to tag the wrapper, exactly as we did with typestate. If you try to write that struct directly, the compiler stops you: an unused type parameter is almost always a mistake, so Rust makes it a hard error rather than a warning.
PhantomData<T> is the fix. It is a zero-sized marker from std::marker that says to the compiler: "pretend I contain a T, even though I store no bytes." It satisfies the "every type parameter must be used" rule without adding a single byte to your struct:
use std::marker::PhantomData;
struct Wrapper<T> {
id: u64,
_marker: PhantomData<T>, // no T is stored, but T now counts as "used"
}
impl<T> Wrapper<T> {
fn new(id: u64) -> Wrapper<T> {
Wrapper { id, _marker: PhantomData }
}
}
struct Meters;
struct Seconds;
fn main() {
let distance: Wrapper<Meters> = Wrapper::new(100);
let duration: Wrapper<Seconds> = Wrapper::new(60);
println!("{} {}", distance.id, duration.id); // 100 60
println!("size = {}", std::mem::size_of::<Wrapper<Meters>>()); // 8, just the u64
}
Look at what we bought. Wrapper<Meters> and Wrapper<Seconds> are two distinct types -- you cannot assign one where the other is expected, cannot mix a distance up with a duration -- which is precisely the typestate and units-of-measure safety from episodes 29 and 30. And yet size_of::<Wrapper<Meters>>() is eight bytes, the size of the u64 alone, because PhantomData<T> contributes nothing. The type distinction lives entirely at compile time; the runtime layout is identical to a struct that never heard of T.
The same idea tags a numeric amount with its unit, which is the heart of the mini project we built in episode 30:
use std::marker::PhantomData;
struct Kg;
struct Lb;
struct Mass<U> {
amount: f64,
_u: PhantomData<U>,
}
fn main() {
let m: Mass<Kg> = Mass { amount: 70.0, _u: PhantomData };
let w: Mass<Lb> = Mass { amount: 154.0, _u: PhantomData };
println!("{} {}", m.amount, w.amount); // 70 154
println!("same size? {}",
std::mem::size_of::<Mass<Kg>>() == std::mem::size_of::<Mass<Lb>>()); // true
}
Mass<Kg> and Mass<Lb> have identical layout -- one f64, eight bytes -- but the compiler treats them as different types and will refuse to let you add a mass in kilograms to a mass in pounds without an explicit conversion. This is the engine behind all of our type-level safety, stated in one line: distinct types at compile time, identical bytes at runtime.
Here is a subtlety worth planting now, because it matters more than it first appears. The exact shape of the type you put inside PhantomData tells the compiler how your struct relates to that type -- whether it behaves as if it owns a T, merely borrows one, or just holds a raw pointer to one. All three are still zero-sized, but they influence things like the auto traits (Send and Sync from episode 25) and how the compiler reasons about the relationship between your type and T:
use std::marker::PhantomData;
// Behaves as if it owns a T (relevant to drop-checking and Send/Sync):
struct Owning<T> {
_m: PhantomData<T>,
}
// Behaves as if it only holds a raw pointer to T (no ownership implied):
struct RawPtr<T> {
_m: PhantomData<*const T>,
}
fn main() {
println!("owning size = {}", std::mem::size_of::<Owning<String>>()); // 0
println!("rawptr size = {}", std::mem::size_of::<RawPtr<String>>()); // 0
}
Both are zero-sized, as promised, so the choice never costs you a byte. But PhantomData<T> versus PhantomData<*const T> versus PhantomData<&'a T> sends the compiler quite different signals about your type's behaviour. The full story of why those signals matter -- and how they change what the compiler will let you substitute for what -- is a topic in its own right, and one we will pick apart properly next time. For now, just file away that the thing inside the brackets is not arbitrary: pick the form that honestly describes what your type does.
PhantomData does not only carry types -- it carries lifetimes too, and this is where it earns its keep in real systems code. Sometimes a value is logically tied to a borrow even though it does not store a reference. The classic example is a handle that is only valid while some backing buffer or arena is alive, but which physically stores a bare integer -- an index or a raw address -- rather than a &. You still want the borrow checker to enforce that the handle cannot outlive the thing it points into. The tool for that is PhantomData<&'a ()>:
use std::marker::PhantomData;
struct Handle<'a> {
raw: usize,
_lifetime: PhantomData<&'a ()>,
}
impl<'a> Handle<'a> {
fn new(raw: usize) -> Handle<'a> {
Handle { raw, _lifetime: PhantomData }
}
}
fn main() {
let handle = Handle::new(0xdead);
println!("{:#x}", handle.raw); // 0xdead
}
The PhantomData<&'a ()> field threads the lifetime 'a into the type. From that point on, the borrow checker treats a Handle<'a> as if it borrowed something for 'a, even though the struct holds nothing but a usize. If you tie a Handle to a local buffer, you get a compile error the moment you try to use the handle after that buffer is gone -- the same protection a real reference would give you, but with the freedom to store a compact integer in stead of a fat pointer. This pattern shows up constantly in FFI wrappers (where you hold an opaque C handle) and in arena allocators (where you hold an index into a slab), which is exactly the kind of territory we will wander into before long.
Step back and notice the theme running through every example above: none of it exists at runtime. The ZST markers add no bytes. PhantomData adds no bytes. The type distinctions and lifetime relationships they create live purely in the compiler's head during type checking, and then evaporate. You can confirm the "adds nothing" claim on even the most substantial-looking phantom:
use std::marker::PhantomData;
struct Plain {
id: u32,
}
struct Tagged {
id: u32,
_p: PhantomData<String>, // looks heavy; weighs nothing
}
fn main() {
println!("plain = {}", std::mem::size_of::<Plain>()); // 4
println!("tagged = {}", std::mem::size_of::<Tagged>()); // 4, PhantomData added nothing
}
PhantomData<String> does not embed a String, does not allocate, does not cost a byte -- Tagged is the same four bytes as Plain. This is why idiomatic Rust reaches for marker types and marker lifetimes so freely when it wants safety. The forementioned alternative -- tagging a value at runtime with an enum discriminant or a boolean flag -- costs memory on every instance and a branch on every check. A phantom type parameter or a marker lifetime costs neither. When you next see PhantomData in someone's code, read it as a plain-English sentence: "this type is tagged with information the compiler enforces but the machine never sees."
Since quite some of you came here from the Learn Python Series, a glance sideways sharpens the point, because Python has no equivalent of any of this. There are no zero-sized types (every object carries a header and lives on the heap), and there is no compile-time type tagging that gets erased before the program runs. If you wanted the units-of-measure safety in Python, you would tag the value at runtime -- typically with a string or a class -- and pay for it in both memory and checking:
class Mass:
def __init__(self, amount, unit):
self.amount = amount
self.unit = unit # a runtime tag: costs memory, checked at runtime
kg = Mass(70, "kg")
lb = Mass(154, "lb")
# Nothing stops you from mixing them until the check actually runs:
if kg.unit != lb.unit:
print("refusing to add different units") # discovered only at runtime
Every Mass in Python drags a unit string around, and the "you cannot mix kilograms and pounds" rule is a runtime if, discovered only when that line executes -- possibly in production, possibly never, if the branch is not exercised by your tests. Rust's version stores no unit tag at all, and the mixing is caught by the compiler, before the program ever runs. Same goal, keep incompatible quantities apart, but Rust turns it into a zero-cost compile-time contract in stead of an always-on runtime chaperone. I argue that is the better bargain for systems code, where you want both the safety and the certainty that it did not cost you anything.
(), field-less structs, and empty structs all qualify. Even a million-element array of a ZST weighs nothing.HashMap<K, ()> -- and that is literally how HashSet is built, because the () values cost no memory.PhantomData<T> is a zero-sized marker that makes an otherwise-unused type parameter "count", giving you distinct types (Wrapper<Meters> vs Wrapper<Seconds>) with identical runtime layout.PhantomData -- T, *const T, &'a T -- signals ownership, pointing, or borrowing to the compiler, and it influences the auto traits. All three are still zero-sized.PhantomData<&'a ()> carries a lifetime into a type, so the borrow checker enforces that a value (holding, say, a raw index) cannot outlive what it points into -- the backbone of safe FFI and arena handles.Three exercises, from gentle to chewier. Type them yourself before the next episode -- that is where the understanding really sticks.
HashMap<String, ()>: insert a few names (with one deliberate duplicate), then print whether it contains one of them and print its length. Confirm the duplicate did not grow the set.Typed<U> wrapper over an f64, with two unit markers Kg and Lb. Construct a Typed<Kg> and a Typed<Lb>, print size_of for each, and confirm the two sizes are equal while the types remain distinct (try assigning one to the other in a commented-out line and note the error you would get).Token<'a> struct holding a usize and a PhantomData<&'a ()> field, plus a function that borrows a local Vec and returns a Token tied to that borrow's lifetime. In a comment, explain what happens if a caller tries to keep the Token alive after the Vec is dropped.Bedankt voor het lezen! ;-)
@scipio