Learn Go Series (#14) - select and the sync Toolbox

scipio(71)

Published in

StemSocial

Words

2847

Reading

13 min

Listen

Play

Learn Go Series (#14) - select and the sync Toolbox

What will I learn

select, which waits on several channels at once and acts on whichever is ready first;
The non-blocking select with a default case, and the timeout pattern with time.After;
sync.Mutex for protecting shared state, and sync.RWMutex for many-readers-one-writer;
sync.Once for run-exactly-once initialisation;
The sync/atomic types for cheap, lock-free counters;
When to reach for channels versus a mutex -- the two honest halves of Go concurrency.

Requirements

A working modern computer running macOS, Windows or Ubuntu;
An installed Go distribution (1.27 or newer, from go.dev/dl) -- tested against Go 1.27;
Episode 13 (goroutines and channels) especially, plus the earlier fundamentals;
The ambition to learn Go programming.

Difficulty

Intermediate

Curriculum (of the `Learn Go Series`):

Learn Go Series (#14) - select and the sync Toolbox

Last episode we built the vocabulary of goroutines and channels. Today we finish the concurrency toolkit with two things. First, select -- the control structure that lets a goroutine wait on several channels at once and respond to whichever fires, which unlocks timeouts, non-blocking checks, and multiplexing. Second, the sync and sync/atomic packages, because sometimes the honest answer really is "lock a shared variable", and Go gives you clean tools for that too. The proverb says "share memory by communicating", but a wise Gopher knows a Mutex is sometimes simpler than a channel.

There is a cultural point hiding in that proverb, and it is worth saying out loud before we write any code. "Share memory by communicating" is excellent advice and the right default -- passing values over channels sidesteps a whole category of locking bugs -- but it was never meant as a prohibition on locks. The Go standard library itself is full of mutexes; the same people who coined the proverb also wrote sync.Mutex and fully expect you to use it. The skill is not "channels good, locks bad", it is knowing which shape fits the problem: reach for a channel when you are handing off work or a value from one goroutine to another, and reach for a mutex when several goroutines are sharing one piece of state that mostly sits still and occasionally gets touched. Todays episode gives you both halves so you can choose honestly in stead of dogmatically. First, last episode's exercises.

Solutions to Episode 13 Exercises

Exercise 1 -- sum with a channel. Each half sums in its own goroutine and sends a partial:

package main

import "fmt"

func main() {
    nums := []int{1, 2, 3, 4, 5, 6}
    mid := len(nums) / 2
    partials := make(chan int, 2)

    go func() {
        s := 0
        for _, n := range nums[:mid] {
            s += n
        }
        partials <- s
    }()
    go func() {
        s := 0
        for _, n := range nums[mid:] {
            s += n
        }
        partials <- s
    }()

    total := <-partials + <-partials // receive both partials and add
    fmt.Println("total:", total)      // 21
}

Exercise 2 -- a pipeline stage. gen launches a goroutine and returns the channel immediately:

package main

import "fmt"

func gen(nums ...int) <-chan int {
    out := make(chan int)
    go func() {
        for _, n := range nums {
            out <- n
        }
        close(out) // signal end of stream
    }()
    return out
}

func main() {
    for v := range gen(1, 2, 3, 4) {
        fmt.Print(v, " ")
    }
    fmt.Println()
}

Exercise 3 -- fix the deadlock. Two independent fixes, buffering versus a live receiver:

package main

import "fmt"

func main() {
    // Fix 1: a buffered channel has room to store the send with no receiver.
    buffered := make(chan int, 1)
    buffered <- 1
    fmt.Println("buffered:", <-buffered)

    // Fix 2: a goroutine receives, giving the unbuffered send a partner.
    unbuffered := make(chan int)
    got := make(chan int)
    go func() { got <- <-unbuffered }() // receive, then forward
    unbuffered <- 2
    fmt.Println("received:", <-got)
    // Buffered STORES the value; unbuffered hands it directly to a live receiver.
}

Now, select.

select waits on many channels

select looks like a switch, but each case is a channel operation. It blocks until one of its cases can proceed, runs that case, and moves on. If several are ready at once, it chooses one at random (so no channel is starved). This is how one goroutine listens to several sources:

package main

import "fmt"

func main() {
    c1 := make(chan string, 1)
    c2 := make(chan string, 1)
    c1 <- "from c1"
    c2 <- "from c2"

    for i := 0; i < 2; i++ {
        select {
        case msg := <-c1:
            fmt.Println(msg)
        case msg := <-c2:
            fmt.Println(msg)
        }
    }
}

Both channels are ready, so each iteration select picks one at random and drains it; over two iterations both are read. The random choice is deliberate -- it prevents a always-ready channel from monopolising the loop. select is the multiplexer of Go concurrency: any time a goroutine must react to whichever of several things happens first, this is the tool.

The mental model that made select click for me is to stop reading it as a switch and start reading it as wait for the first of these to become possible. A switch evaluates one expression and branches; a select sits on all of its cases simultaneously and wakes up the instant any one of them can send or receive. That is a genuinely different shape of control flow, and it is the shape almost every long-lived goroutine ends up wanting: a loop with a select at the top that reacts to work arriving on one channel, a shutdown signal on another, maybe a timer on a third. Having said that, watch the random-choice rule closely, because it has a practical consequence -- you cannot rely on priority between cases. If you need "drain the urgent channel first, only then look at the normal one", a single select will not give you that; you nest selects, or you check the priority channel with a non-blocking select before falling through to the blocking one. For the common case, though, the fairness is a gift: no well-behaved sender can starve the others just by being fast.

Non-blocking select with default

Add a default case and select stops blocking: if no channel operation is ready right now, default runs immediately. This turns a channel check into a non-blocking poll -- "is there anything waiting? no? carry on":

package main

import "fmt"

func main() {
    ch := make(chan int) // empty, and nobody is sending

    select {
    case v := <-ch:
        fmt.Println("received", v)
    default:
        fmt.Println("nothing ready, moving on") // runs because no case is ready
    }
}

Without default, the receive would block forever (deadlock, since nothing sends). With it, the select falls through instantly. Use default for "try this channel, but do not wait" -- draining without blocking, or a periodic check inside a loop. Do not put a bare default inside a tight for loop with nothing else, though, or you spin the CPU at 100 percent.

That last warning is worth taking seriously, because the non-blocking select is the one concurrency tool that will happily waste an entire core while looking like it is doing something. A for { select { ...; default: } } with nothing to block on is a busy-wait: it loops millions of times a second asking "anything yet? anything yet?" and pins the CPU for no benefit at all. The default case earns its keep only when you have real work to do between checks -- poll the channel, and if nothing is there, get on with the other thing you were going to do anyway. If the honest answer is "I have nothing to do until a channel is ready", then you do not want default; you want a plain blocking select, which parks the goroutine at zero cost until something actually happens. Nota bene: the runtime scheduler is very good at waking a parked goroutine the instant its channel becomes ready, so blocking is almost always cheaper and simpler than polling. Reaching for default should feel like a deliberate choice, not a reflex.

Timeouts with time.After

The single most useful select pattern is the timeout. time.After(d) returns a channel that delivers a value after duration d. Race your real operation against it, and whichever fires first wins -- so a slow operation cannot hang you forever:

package main

import (
    "fmt"
    "time"
)

func main() {
    slow := make(chan string)
    go func() {
        time.Sleep(50 * time.Millisecond)
        slow <- "eventually"
    }()

    select {
    case msg := <-slow:
        fmt.Println("got:", msg)
    case <-time.After(200 * time.Millisecond):
        fmt.Println("timed out")
    }
}

Here the work finishes in 50ms, comfortably inside the 200ms deadline, so we print "got: eventually". Shrink the deadline below 50ms and the timeout case wins instead. This is the seed of the deadline handling we do properly with context next episode -- time.After is the low-level version, and context is the version you actually use in real programs.

One subtlety about time.After bites people once they start using it inside loops: each call creates a new timer, and that timer is not garbage-collected until it fires, even if your real operation already won the race. In a select that runs once, like the one above, this is a complete non-issue -- the timer fires (or is collected) shortly after and you move on. But drop time.After into a for loop that iterates thousands of times a second and you can pile up thousands of pending timers, each holding a sliver of memory until its deadline passes. The fix, when you get there, is to create one time.Timer with time.NewTimer and Reset it each iteration -- or, far more commonly, to let context carry the deadline for you, which is exactly the upgrade next episode delivers. For a one-shot timeout, though, time.After is perfectly clean and reads beautifully, so use it without guilt where it fits.

sync.Mutex: protect shared state

When goroutines genuinely share a variable, a sync.Mutex gives one exclusive access at a time. Lock before touching the shared data, Unlock after -- and defer the unlock so it always happens. Embedding the mutex next to the data it guards is the idiomatic shape:

package main

import (
    "fmt"
    "sync"
)

type Counter struct {
    mu    sync.Mutex
    count int
}

func (c *Counter) Inc() {
    c.mu.Lock()
    defer c.mu.Unlock()
    c.count++ // only one goroutine is ever in here at a time
}

func main() {
    c := &Counter{}
    var wg sync.WaitGroup
    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            c.Inc()
        }()
    }
    wg.Wait()
    fmt.Println("count:", c.count) // exactly 1000, no race
}

A thousand goroutines increment, and because Inc holds the lock while touching count, the result is exactly 1000 every time. Remove the lock and you have a data race -- concurrent unsynchronised access, the result unpredictable, the bug intermittent. (Next episode's neighbour, the -race flag, catches exactly this.) The rule: any variable touched by more than one goroutine, where at least one writes, needs synchronisation -- a mutex or a channel.

Two habits make mutexes boring in the good way, and boring is exactly what you want from your locking. The first is to defer the Unlock immediately after the Lock, on the very next line, so the unlock is impossible to forget no matter how many return paths the function grows later -- the same defer discipline we used for closing files back in the fundamentals. The second, and the one that pays off most, is to keep the critical section small: lock, touch the shared data, unlock, and do everything else -- the slow computation, the logging, the I/O -- outside the lock. A mutex held across a slow operation serialises every other goroutine behind it, and you have quietly turned your concurrent program back into a sequential one with extra overhead on top. Keep the guarded region to the few lines that actually read or write the shared state, and put the mutex right next to the data it protects (embedded in the struct, as above) so it is obvious at a glance what the lock covers. A lock whose scope you have to guess is a bug patiently waiting to happen.

sync.RWMutex and sync.Once

Two more sync tools earn their place often. A sync.RWMutex distinguishes readers from writers: any number of readers can hold the read lock at once, but a writer needs exclusive access -- a win when reads vastly outnumber writes. And sync.Once runs a function exactly once, no matter how many goroutines call it, which is the clean way to do lazy, thread-safe initialisation:

package main

import (
    "fmt"
    "sync"
)

type SafeMap struct {
    mu sync.RWMutex
    m  map[string]int
}

func (s *SafeMap) Get(k string) int {
    s.mu.RLock() // shared read lock: many readers at once
    defer s.mu.RUnlock()
    return s.m[k]
}

func (s *SafeMap) Set(k string, v int) {
    s.mu.Lock() // exclusive write lock
    defer s.mu.Unlock()
    s.m[k] = v
}

func main() {
    sm := &SafeMap{m: make(map[string]int)}
    sm.Set("x", 42)
    fmt.Println("x =", sm.Get("x")) // 42
}

RLock/RUnlock for reads, Lock/Unlock for writes -- exactly the protection a shared map needs, since Go's built-in map is not safe for concurrent writes. And here is sync.Once guaranteeing one-time setup:

package main

import (
    "fmt"
    "sync"
)

var (
    once     sync.Once
    instance string
)

func getInstance() string {
    once.Do(func() {
        instance = "the one instance"
        fmt.Println("(expensive init running)")
    })
    return instance
}

func main() {
    fmt.Println(getInstance())
    fmt.Println(getInstance()) // init does NOT run a second time
}

once.Do(f) runs f on the first call and never again, even across goroutines racing to call it -- the standard, race-free way to initialise a singleton or a shared resource lazily.

A word of honesty about RWMutex, because it is easy to over-reach for it: a read-write lock is only a win when reads genuinely dominate and the critical section is long enough that the extra bookkeeping pays for itself. For a couple of quick field reads a plain Mutex is often faster, because RWMutex does more work internally to track readers versus writers. Measure before you assume the "many readers" version is quicker -- sometimes the simpler lock wins, and you would be surprised how often. sync.Once, on the other hand, I reach for without hesitation whenever I have lazy initialisation that must happen exactly once: it is race-free by construction, it makes concurrent callers block until the first one finishes the setup (so nobody ever sees a half-initialised value), and it reads its own intent right there in the name. It is the honest replacement for the tempting-but-broken "check a boolean, then initialise if it is false" pattern, which has a race hiding between the check and the set -- a race that Once closes for you completely.

sync/atomic: lock-free counters

For simple numeric operations, a full mutex is more than you need. The sync/atomic package provides atomic types whose operations are indivisible at the hardware level -- perfect for counters and flags shared across goroutines, with less overhead than a lock:

package main

import (
    "fmt"
    "sync"
    "sync/atomic"
)

func main() {
    var counter atomic.Int64 // a concurrency-safe integer

    var wg sync.WaitGroup
    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            counter.Add(1) // atomic increment, no lock needed
        }()
    }
    wg.Wait()
    fmt.Println("count:", counter.Load()) // 1000
}

atomic.Int64 (and its siblings atomic.Bool, atomic.Int32, atomic.Pointer) offer Add, Load, Store, Swap, and CompareAndSwap, each a single uninterruptible operation. Reach for atomics when the shared state is one number or flag; reach for a mutex when you need to guard a compound update or a whole data structure. Both are correct -- atomics are just leaner for the narrow case.

The line I draw in practice is about how much state moves together. An atomic operation is indivisible for one value -- one integer, one flag, one pointer -- and that is its whole domain. The moment your invariant spans two variables ("increment this counter and append to that slice, and a reader must never see one without the other"), atomics cannot help you, because there is no such thing as atomically updating two independent locations at once; you need a mutex to make the pair look like a single indivisible change. So the heuristic stays simple: one number or flag, reach for sync/atomic; a compound update or a whole structure, reach for a sync.Mutex. And do not mix the two on the same field -- a value guarded by a mutex should be touched only under that mutex, and an atomic value only through atomic operations, never with a plain = on the side when you think nobody is looking. That consistency is what keeps either tool correct; break it and you have the worst of both worlds, a race the compiler cannot see.

The same idea in Python

Python's threading gives you Lock, RLock, and Event, and asyncio has its own primitives, but there is no direct select-over-channels in the standard library -- you would reach for queue with timeouts, or asyncio.wait:

import threading

class Counter:
    def __init__(self):
        self._lock = threading.Lock()
        self.count = 0
    def inc(self):
        with self._lock:   # like defer mu.Unlock(), the with-block releases
            self.count += 1

c = Counter()
threads = [threading.Thread(target=c.inc) for _ in range(1000)]
for t in threads: t.start()
for t in threads: t.join()
print(c.count)             # 1000

Python's with lock: mirrors Go's Lock/defer Unlock, and Python threads are real -- but the GIL serialises CPU-bound Python code, so threads mainly help with I/O. Go's goroutines run truly in parallel across cores, and select gives it a channel-multiplexing construct Python has no direct equal to. Between channels for coordination and sync/atomics for shared state, you now have the whole everyday concurrency toolkit.

The deeper point behind the comparison is where the coordination machinery lives. In Python you assemble it from library pieces -- a Lock here, a Queue with a timeout there, an Event to signal shutdown -- and you write the protocol that ties them together yourself, including the awkward bits like "put one sentinel per consumer on the queue so every worker sees the stop". Go folds that protocol into the language: select is a keyword, close broadcasts to every receiver at once, range over a channel stops itself when the channel closes, and the scheduler runs goroutines across real OS threads so the same code that overlaps I/O also parallelises CPU work. Neither approach is wrong -- Python's model is perfectly serviceable, and its GIL trade-off buys real simplicity elsewhere -- but Go's bet is that concurrency is common enough to deserve first-class syntax, and once you have lived in it for a while the difference is hard to unsee. That is the whole toolkit now: channels and select for coordination, sync and atomics for the honest cases where you really are just guarding a piece of shared state.

Exercises

A timeout wrapper. Write func fetch(delay, timeout time.Duration) (string, error) that starts a goroutine which sleeps delay and then sends a result, and uses select with time.After(timeout) to return either the result or an error. Call it once where it succeeds and once where it times out.
A concurrent-safe counter, two ways. Implement the same "increment from 1000 goroutines" once with a sync.Mutex and once with atomic.Int64, and confirm both print 1000. Note in a comment which you would prefer for a single counter and why.
Fan-in with select. Given two channels each producing a few integers (from goroutines that close them when done), write a loop that uses select to read from whichever is ready and prints values until both are closed. (Hint: set a channel variable to nil once it is closed -- a nil channel case in a select is never chosen.)

What we learned

select waits on several channel operations and runs whichever is ready, choosing at random among ties -- the multiplexer of Go concurrency;
A default case makes select non-blocking (poll and move on); racing an operation against time.After(d) gives you a timeout;
sync.Mutex grants one goroutine exclusive access to shared state (Lock/defer Unlock); unsynchronised shared access is a data race;
sync.RWMutex lets many readers or one writer proceed; sync.Once runs an initialiser exactly once across all goroutines;
sync/atomic types (atomic.Int64 and friends) give lock-free, indivisible operations -- ideal for a shared counter or flag;
Channels for coordination, mutexes/atomics for shared state: both are idiomatic, and picking the simpler one for the job is the mark of good Go.

Next episode ties concurrency to the real world with context: how to cancel a goroutine, propagate a deadline through a call tree, and carry request-scoped values -- the piece every Go server and client leans on. See you there.

Bedankt en tot de volgende keer! ;-)

@scipio