Deref, DerefMut and Drop;Box::into_raw, Box::from_raw and NonNull let you own a raw allocation by hand;Clone and Drop cooperate to keep a reference count exactly correct;Box and Rc you have leaned on all series actually work on the inside.Deref (episode 19), Drop (episode 20), interior mutability (episode 32) and Rc internals (episode 33);Learn Rust Series):You have used Box, Rc and RefCell throughout this whole series, and in episodes 12, 33 and 34 we opened the hood far enough to see how they behave. Today we go one step further and actually build our own, because nothing cements understanding quite like implementing the thing you have been relying on. Last episode I promised that all the low-level machinery we assembled -- raw pointers, marker fields, deciding precisely how a type relates to its parameters -- was groundwork for the moment we start hand-building allocators and pointer types. Episode 40 was the allocator half of that promise. This is the pointer half.
We will start gently, with a wrapper that participates in dereferencing and cleanup, and then build a genuine reference-counted pointer from a raw heap allocation. That second one needs unsafe, and seeing exactly where the unsafe lives and why it is nonetheless sound is the real lesson of the day ;-)
Episode 40 was arenas and bump allocation. There were three exercises, and here is full, runnable code for each.
Exercise 1 asked you to extend the pure-std index Arena<T> with an iter(&self) method returning an iterator over &T for every allocated item, and to use it to print a small arena:
struct Arena<T> {
items: Vec<T>,
}
impl<T> Arena<T> {
fn new() -> Arena<T> {
Arena { items: Vec::new() }
}
fn alloc(&mut self, value: T) -> usize {
let index = self.items.len();
self.items.push(value);
index
}
fn iter(&self) -> std::slice::Iter<'_, T> {
self.items.iter() // the Vec's own iterator IS the arena's iterator
}
}
fn main() {
let mut arena = Arena::new();
arena.alloc("hello");
arena.alloc("world");
for value in arena.iter() {
println!("{value}"); // hello, then world
}
}
The hint gave the whole game away: because the arena is a Vec underneath, self.items.iter() already yields exactly &T per element. The interesting bit is the return type -- std::slice::Iter<'_, T> names the concrete iterator a slice hands back, and the '_ lets the compiler tie its lifetime to &self. You could also have written -> impl Iterator<Item = &T> + '_, which hides the concrete type behind the trait; both are correct, and which you pick is a small API-design choice about how much you want to expose.
Exercise 2 wanted a depth(&self, i: usize) -> usize method on the expression Tree that returns the height of the sub-tree rooted at index i -- a Num has depth 1, an Add or Mul is 1 plus the larger of its two children's depths -- tested on (2 + 3) * 4:
enum Expr {
Num(i32),
Add(usize, usize),
Mul(usize, usize),
}
struct Tree {
nodes: Vec<Expr>,
}
impl Tree {
fn depth(&self, i: usize) -> usize {
match self.nodes[i] {
Expr::Num(_) => 1,
Expr::Add(l, r) | Expr::Mul(l, r) => 1 + self.depth(l).max(self.depth(r)),
}
}
}
fn main() {
let t = Tree {
nodes: vec![
Expr::Num(2), // 0
Expr::Num(3), // 1
Expr::Add(0, 1), // 2 = (2 + 3)
Expr::Num(4), // 3
Expr::Mul(2, 3), // 4 = (2 + 3) * 4
],
};
println!("{}", t.depth(4)); // 3
}
Notice the Expr::Add(l, r) | Expr::Mul(l, r) or-pattern: since both variants carry two child indices and the recurrence is identical, one arm handles both, and this is a small taste of the pattern matching we met back in episode 4. Trace it: depth(4) is a Mul over indices 2 and 3, so it is 1 + max(depth(2), depth(3)). depth(2) is the Add, which is 1 + max(depth(0), depth(1)) = 1 + max(1, 1) = 2, and depth(3) is a Num, so 1. The result is 1 + max(2, 1) = 3, exactly the height you would count by eye.
Exercise 3 asked you to take the free-list Arena<T> and add a get(&self, i: usize) -> Option<&T> method that returns None for a slot that is currently free and Some(&value) for a live one:
struct Arena<T> {
items: Vec<Option<T>>,
free: Vec<usize>,
}
impl<T> Arena<T> {
fn new() -> Arena<T> {
Arena { items: Vec::new(), free: Vec::new() }
}
fn alloc(&mut self, v: T) -> usize {
if let Some(i) = self.free.pop() {
self.items[i] = Some(v);
i
} else {
self.items.push(Some(v));
self.items.len() - 1
}
}
fn free(&mut self, i: usize) {
self.items[i] = None;
self.free.push(i);
}
fn get(&self, i: usize) -> Option<&T> {
self.items[i].as_ref() // Option -> Option, None stays None
}
}
fn main() {
let mut a = Arena::new();
let x = a.alloc("first");
let y = a.alloc("second");
a.free(x);
println!("freed slot live? {}", a.get(x).is_some()); // false
println!("other slot live? {}", a.get(y).is_some()); // true
}
The whole method is self.items[i].as_ref(). That Option::as_ref turns an &Option<T> into an Option<&T> without moving anything out of the arena -- a freed slot holds None, so get naturally reports it as empty, and a live slot holds Some(value), so you get a borrow of it. Right, homework cleared. Now let us build a pointer of our own.
Before we write a line, let me pin down the definition, because it is refreshingly small. A smart pointer is any type that owns some data and implements the traits that let it behave like a pointer to that data. There is no magic keyword and no special compiler blessing; the "smartness" is entirely a matter of which traits you implement. Three of them do almost all the work, and we have met every one already:
Deref (episode 19) gives you *p for reading through the pointer, and it is what powers deref coercion, the quiet feature that lets &Box<String> be used where &str is expected.DerefMut (also episode 19) gives you *p = ... and &mut *p for writing through it.Drop (episode 20) gives you a destructor that runs deterministically when the pointer goes out of scope -- the RAII cleanup hook.Implement those, and the type you wrote is indistinguishable, at the call site, from a built-in pointer. That is the entire trick. Everything Box and Rc do beyond it is a matter of what they own (a heap allocation, or a heap allocation plus a shared count) and how they clean it up.
Let us make the smallest possible one. It just wraps a value inline -- no heap yet -- but it participates fully in dereferencing and cleanup:
use std::ops::{Deref, DerefMut};
struct MyBox<T> {
value: T,
}
impl<T> MyBox<T> {
fn new(value: T) -> MyBox<T> {
MyBox { value }
}
}
impl<T> Deref for MyBox<T> {
type Target = T;
fn deref(&self) -> &T {
&self.value
}
}
impl<T> DerefMut for MyBox<T> {
fn deref_mut(&mut self) -> &mut T {
&mut self.value
}
}
impl<T> Drop for MyBox<T> {
fn drop(&mut self) {
println!("MyBox is being dropped");
}
}
fn main() {
let mut b = MyBox::new(10);
*b += 5; // DerefMut lets us mutate through the pointer
println!("{}", *b); // Deref lets us read: 15
} // "MyBox is being dropped" prints here, at the closing brace
Read it slowly, because every line earns its place. *b reads through Deref, so it desugars to *(b.deref()). *b += 5 needs to write, so the compiler reaches for DerefMut in stead and desugars to *(b.deref_mut()) += 5. And when b falls out of scope at the closing brace, Drop::drop runs, printing the message. That is a fully functional smart pointer in about twenty lines. The real Box differs in exactly one respect: its new heap-allocates the value and its drop frees that allocation. The interface -- the three traits -- is the same.
The Deref implementation is worth dwelling on for a moment, because it buys you more than *b. Recall from episode 19 that when you pass &MyBox<T> where a &T (or even a &U reachable by a chain of derefs) is expected, the compiler inserts the deref calls for you automatically. This is why you can call String methods on a Box<String> without ever thinking about it, and our MyBox inherits the same courtesy:
use std::ops::Deref;
struct MyBox<T> {
value: T,
}
impl<T> MyBox<T> {
fn new(value: T) -> MyBox<T> {
MyBox { value }
}
}
impl<T> Deref for MyBox<T> {
type Target = T;
fn deref(&self) -> &T {
&self.value
}
}
fn greet(name: &str) {
println!("Hello, {name}!");
}
fn main() {
let b = MyBox::new(String::from("Rust"));
greet(&b); // &MyBox -> &String -> &str, two coercions inserted for free
println!("length via Deref: {}", b.len()); // String::len reached straight through the pointer
}
Passing &b to a function wanting &str works because the compiler quietly walks &MyBox<String> to &String (our Deref) and then &String to &str (the standard library's Deref on String), stacking two coercions in a row. Likewise b.len() finds no len on MyBox, so it derefs and finds String::len. None of this is special-cased for the standard library -- it falls straight out of the one Deref impl we wrote. Having said that, a value wrapper is a bit of a cheat: a real box puts its payload on the heap.
To make MyBox a genuine box that owns a heap allocation, the simplest honest version stores a real Box<T> inside and forwards through it. It still derefs cleanly, and it now really does live on the heap:
use std::ops::Deref;
struct HeapBox<T> {
inner: Box<T>,
}
impl<T> HeapBox<T> {
fn new(v: T) -> HeapBox<T> {
HeapBox { inner: Box::new(v) } // the value is moved onto the heap here
}
}
impl<T> Deref for HeapBox<T> {
type Target = T;
fn deref(&self) -> &T {
&self.inner
}
}
fn main() {
let b = HeapBox::new(vec![1, 2, 3]);
println!("len through two derefs: {}", b.len()); // Vec::len via HeapBox -> Box -> Vec
}
We are cheating slightly by leaning on the standard Box to do the actual allocating and freeing for us -- when HeapBox drops, its inner: Box<T> drops in turn and frees the allocation, no explicit Drop needed. That is a perfectly respectable way to build a wrapper, and the newtype pattern from episode 24 is exactly this shape. But it dodges the question I really want to answer today: what does it take to own a raw heap allocation by hand, with no Box doing the hard part? For that, we build a reference counter.
Now the interesting one. We are going to build a reference-counted pointer, a stripped-down Rc, from a raw heap allocation. The plan mirrors exactly what we reverse-engineered in episode 33: allocate one small heap box holding both a count and the value, share a raw pointer to it across every clone, bump the count up on clone, and on drop either decrement the count or -- if we are the last owner -- free the allocation.
use std::cell::Cell;
use std::ops::Deref;
use std::ptr::NonNull;
struct RcBox<T> {
count: Cell<usize>, // interior mutability: we mutate this through a shared &
value: T,
}
struct MyRc<T> {
ptr: NonNull<RcBox<T>>,
}
impl<T> MyRc<T> {
fn new(value: T) -> MyRc<T> {
let boxed = Box::new(RcBox { count: Cell::new(1), value });
// Box::into_raw hands us ownership of the allocation as a raw pointer.
// From here on, WE are responsible for freeing it -- the Box no longer will.
MyRc { ptr: NonNull::new(Box::into_raw(boxed)).unwrap() }
}
fn count(&self) -> usize {
// SAFETY: ptr is valid as long as at least one MyRc exists, and self is one.
unsafe { self.ptr.as_ref().count.get() }
}
}
impl<T> Clone for MyRc<T> {
fn clone(&self) -> MyRc<T> {
// SAFETY: same live allocation; we only bump the shared count.
unsafe {
let c = self.ptr.as_ref().count.get();
self.ptr.as_ref().count.set(c + 1);
}
MyRc { ptr: self.ptr } // NonNull is Copy, so we just duplicate the pointer
}
}
impl<T> Deref for MyRc<T> {
type Target = T;
fn deref(&self) -> &T {
// SAFETY: the allocation lives as long as any MyRc does, self included.
unsafe { &self.ptr.as_ref().value }
}
}
impl<T> Drop for MyRc<T> {
fn drop(&mut self) {
// SAFETY: the last owner reclaims the Box and frees it; others only decrement.
unsafe {
let c = self.ptr.as_ref().count.get();
if c == 1 {
drop(Box::from_raw(self.ptr.as_ptr())); // rebuild the Box, then free it
} else {
self.ptr.as_ref().count.set(c - 1);
}
}
}
}
fn main() {
let a = MyRc::new(String::from("shared"));
println!("count = {}", a.count()); // 1
let b = a.clone();
println!("count = {}", a.count()); // 2
println!("value = {}", *b); // shared
drop(b);
println!("count = {}", a.count()); // 1
} // a drops here, count would hit 0, so the allocation is freed
Let me trace the lifecycle, because this is the heart of the episode. new builds an RcBox on the heap with Box::new, and then Box::into_raw consumes that Box and gives us back the raw pointer to its allocation. That call is the pivotal moment: from there on the Box is gone and we own the memory, meaning we and we alone are on the hook to free it eventually. We wrap the raw pointer in NonNull, which is just a raw pointer that is guaranteed never to be null (and, as a bonus from episode 39, gives us the covariance we want). clone copies the pointer -- cheap, it is only a machine word -- and bumps the shared count. deref hands out a reference into the one allocation. And drop reads the count: if it is 1 we are the final owner, so Box::from_raw rebuilds a Box from the raw pointer and dropping that Box runs the value's destructor and frees the memory; otherwise we simply decrement and walk away. That is Rc, minus the weak count from episode 35, in roughly forty lines.
One detail that is easy to skate past: why is count a Cell<usize> in stead of a plain usize? Because every handle only ever holds a shared reference to the RcBox (that is the entire point -- shared ownership), and you cannot mutate through a shared reference without interior mutability. Cell is exactly the tool episode 32 gave us for that. The real Rc uses an UnsafeCell under the hood for the same reason, and Arc (episode 34) swaps in an atomic integer so the bump is thread-safe.
Now, the part that matters most. Every raw-pointer access sits inside an unsafe block carrying a SAFETY comment, and that is not decoration -- it is the contract you are signing. The compiler cannot verify that self.ptr still points at a live RcBox, because raw pointers carry no lifetime and it has no way to track them. So the obligation shifts to you: you must guarantee the invariant that the compiler cannot.
And we do uphold it, by construction. The allocation is born in new with a count of 1. It stays alive as long as any MyRc exists, because the count tracks exactly the number of live handles -- every clone increments, every drop decrements. And it is freed precisely once, in the drop of the last handle, when the count would fall to zero. There is no code path that reads self.ptr after the free, because the only handle that frees is the one being dropped, and a dropped handle is never touched again. That is what "sound unsafe" means: a small, auditable core of unsafe, wrapped in a completely safe public API that no caller can misuse. Users of MyRc never write unsafe, exactly as users of Rc never do. You can watch the free happen at exactly the right instant by wrapping a value whose Drop prints:
use std::cell::Cell;
use std::ptr::NonNull;
struct RcBox<T> {
count: Cell<usize>,
value: T,
}
struct MyRc<T> {
ptr: NonNull<RcBox<T>>,
}
impl<T> MyRc<T> {
fn new(v: T) -> MyRc<T> {
MyRc {
ptr: NonNull::new(Box::into_raw(Box::new(RcBox { count: Cell::new(1), value: v }))).unwrap(),
}
}
}
impl<T> Clone for MyRc<T> {
fn clone(&self) -> MyRc<T> {
unsafe {
let c = self.ptr.as_ref().count.get();
self.ptr.as_ref().count.set(c + 1);
}
MyRc { ptr: self.ptr }
}
}
impl<T> Drop for MyRc<T> {
fn drop(&mut self) {
unsafe {
let c = self.ptr.as_ref().count.get();
if c == 1 {
drop(Box::from_raw(self.ptr.as_ptr()));
} else {
self.ptr.as_ref().count.set(c - 1);
}
}
}
}
struct Tracked;
impl Drop for Tracked {
fn drop(&mut self) {
println!("Tracked value freed");
}
}
fn main() {
let a = MyRc::new(Tracked);
let b = a.clone();
drop(a); // count 2 -> 1, nothing is freed yet
println!("still alive after dropping one handle");
drop(b); // count 1 -> 0, "Tracked value freed" prints exactly here
}
Run it in your head: a creates the allocation, b clones it to count 2. Dropping a takes the count to 1 and frees nothing -- and crucially, "still alive" prints before any free message. Only when b drops does the count reach zero and Tracked's destructor finally run. The output proves the counting is exact: the resource is released at the last handle and not a moment sooner. This is the behaviour you have trusted Rc to provide since episode 12, now visibly ours.
A smart pointer is a normal type, so nothing stops you from giving it inherent methods beyond the trait impls. A pleasant one is a map that transforms the owned value into a fresh wrapper, echoing the Option::map and iterator adapters from episode 11:
struct MyBox<T> {
value: T,
}
impl<T> MyBox<T> {
fn new(v: T) -> MyBox<T> {
MyBox { value: v }
}
fn map<U, F: FnOnce(T) -> U>(self, f: F) -> MyBox<U> {
MyBox::new(f(self.value)) // consume self, apply f, rewrap the result
}
}
fn main() {
let b = MyBox::new(5)
.map(|n| n * 2) // MyBox -> MyBox
.map(|n| n.to_string()); // MyBox -> MyBox
println!("{}", b.value); // 10
}
Because map takes self by value and the closure FnOnce(T) -> U, it can change the wrapped type entirely, letting you chain transformations that end up at a different type than they started -- here i32 all the way to String. That is exactly the shape of the combinator methods littered across the standard library, and building one yourself demystifies the lot of them.
Since quite some of you came here from the Learn Python Series, a sideways glance helps. In Python, every object is effectively a reference-counted smart pointer already, and you never see the machinery -- CPython bumps a refcount on every assignment and frees the object when it hits zero, exactly like our MyRc, just hidden inside the interpreter and paid for on every single variable binding. It even runs a backup cycle collector to mop up the reference cycles that plain counting leaks, which is precisely the problem episode 35 solved explicitly with Weak. Rust makes you ask for reference counting with Rc, so you only pay for it where you actually want shared ownership, and everything else stays a zero-overhead move.
In C++, our MyRc is essentially a hand-rolled std::shared_ptr, and the parallels are exact: a control block holding the count, a raw pointer shared across copies, an atomic increment for the thread-safe variant. The difference is the guardrail. In C++ nothing stops you from dereferencing a shared_ptr you already moved from, or from mixing up raw and smart pointers into a double free; in Rust the safe public API makes those mistakes unrepresentable, and the only place a bug could hide is the small unsafe core you can audit on one screen. And in Go, you would simply not build this at all -- the garbage collector owns every heap object and you never think about frees, trading the predictable, immediate cleanup our Drop gives us for the convenience of a tracing collector and its pauses. Three languages, three points on the same spectrum: Rust is the one that hands you the raw mechanism and the safety, and asks you to be explicit about which you want.
Deref (read through *), DerefMut (write through *) and usually Drop (deterministic cleanup). There is no keyword -- the traits are the smartness.MyBox shows the whole interface in twenty lines, and inherits deref coercion for free: &MyBox<String> flows to &str through stacked Deref calls the compiler inserts.NonNull raw pointer across clones, and uses Box::into_raw / Box::from_raw to take and give back ownership of that allocation by hand.Cell because every handle only holds a shared reference -- interior mutability (episode 32) is what lets us bump it. Arc swaps in an atomic for the same reason.unsafe is small, contained and sound: raw-pointer access the compiler cannot verify, wrapped in a safe API whose invariant (count equals live-handle count) we uphold by construction. That is the model for all good unsafe code.shared_ptr without Rust's guardrails; Rust gives you the raw mechanism and makes misuse unrepresentable.Three exercises, from gentle to chewier. Type them yourself before the next episode -- pointer types are exactly the kind of thing that only truly clicks once you have watched your own count tick up and down.
strong_count(this: &MyRc<T>) -> usize associated function to MyRc, matching the signature style of the real Rc::strong_count (an associated function, called as MyRc::strong_count(&a), not a method). Have it return the current count, and confirm it reads the same value as calling .count().MyRc a ptr_eq(&self, other: &MyRc<T>) -> bool that reports whether two handles point at the same allocation, by comparing their raw pointers with NonNull::as_ptr. Confirm it returns true for a handle and its clone, and false for two independently MyRc::new-ed values holding equal data.try_unwrap(self) -> Result<T, MyRc<T>> to MyRc that returns Ok(value) when the count is exactly 1 (you are the sole owner) and Err(self) otherwise. This is trickier than it looks: when the count is 1 you must move the value out of the allocation and free the box without running MyRc's own Drop on self -- reach for std::mem::forget after you have reclaimed the value, and think carefully about why that is necessary.That third one is a deliberate cliff-hanger: getting a value out of a smart pointer without tripping its destructor is the exact problem that the next stretch of episodes circles around. Chew on it ;-)
Bedankt voor het meebouwen, en tot de volgende keer!
@scipio