Learn Go Series (#12) - Packages, Modules, and Project Layout

scipio(71)

Published in

StemSocial

Words

3076

Reading

14 min

Listen

Play

10h

Learn Go Series (#12) - Packages, Modules, and Project Layout

What will I learn

What a package is, and how a name's capitalization decides whether it is exported or private;
What a module is, what go.mod records, and the everyday go commands that manage it;
How to split a program into packages and lay out a real project directory;
The special internal/ directory that enforces "you cannot import this from outside";
Import forms: grouping, aliases, and the blank import for side effects;
init functions, package-level variables, and multi-module workspaces with go.work.

Requirements

A working modern computer running macOS, Windows or Ubuntu;
An installed Go distribution (1.27 or newer, from go.dev/dl) -- tested against Go 1.27;
Episodes 1-11 -- this one ties the fundamentals together into real projects;
The ambition to learn Go programming.

Difficulty

Intermediate

Curriculum (of the `Learn Go Series`):

Learn Go Series (#12) - Packages, Modules, and Project Layout

We have written a lot of small programs; today we learn to organise a big one. Go's answer to "how do I structure a project" is refreshingly mechanical: code lives in packages, a name is public or private based purely on its first letter, and a module (a go.mod file) ties a set of packages together with a name and a version. There is no elaborate build configuration and no separate declaration of what is public -- the language and the directory structure carry it.

I have spent more hours than I would like to admit fighting other languages' project tooling -- config files that describe the build, other config files that describe the packaging, and a third set that lists what is exported and from where. Go throws almost all of that away, and the first time you feel it is a small relief: the directory tree is the package structure, the capital letter is the access modifier, and one go.mod file is the dependency manifest. Less to configure means less to get wrong, and it means that when you open an unfamiliar Go repository you already know how to read it. That uniformity is not an accident -- it is a deliberate design goal, the same one that gave us gofmt and a single canonical formatting. This episode closes out the fundamentals; from next episode we start building real things. First, last episode's exercises.

Solutions to Episode 11 Exercises

Exercise 1 -- wrap and match. Wrap a sentinel, detect it with errors.Is:

package main

import (
    "errors"
    "fmt"
)

var ErrTooShort = errors.New("password too short")

func validatePassword(p string) error {
    if len(p) < 8 {
        return fmt.Errorf("validatePassword: %w", ErrTooShort)
    }
    return nil
}

func main() {
    if err := validatePassword("abc"); errors.Is(err, ErrTooShort) {
        fmt.Println("please use at least 8 characters")
    }
}

Exercise 2 -- a custom error with data. Pull it back out with errors.As:

package main

import (
    "errors"
    "fmt"
)

type HTTPError struct {
    Code    int
    Message string
}

func (e *HTTPError) Error() string {
    return fmt.Sprintf("%d: %s", e.Code, e.Message)
}

func fetch() error {
    return fmt.Errorf("fetch: %w", &HTTPError{Code: 404, Message: "not found"})
}

func main() {
    err := fetch()
    var he *HTTPError
    if errors.As(err, &he) {
        fmt.Println("status code:", he.Code) // 404
    }
}

Exercise 3 -- handle, do not panic. Check the precondition, return an error:

package main

import (
    "errors"
    "fmt"
)

func safeDivide(a, b int) (int, error) {
    if b == 0 {
        return 0, errors.New("division by zero")
    }
    return a / b, nil
}

func main() {
    if _, err := safeDivide(1, 0); err != nil {
        fmt.Println("error:", err)
    }
    // Idiomatic: check b == 0 and return an error, rather than letting the
    // runtime panic and then recovering from it.
}

Now, structure.

Packages, and the capital-letter rule

Every Go file starts with package <name>, and all files in one directory belong to the same package. The rule for visibility could not be simpler: a name starting with a capital letter is exported -- visible to code in other packages -- and a lowercase name is private to its own package. There is no public/private keyword; the case is the modifier:

package main

import "fmt"

// A Capitalized name is EXPORTED; a lowercase name is package-private.
type User struct {
    Name  string // exported: other packages can read/write this
    email string // unexported: only this package can touch it
}

func (u User) Email() string { return u.email } // exported accessor for a private field

func main() {
    u := User{Name: "Ada", email: "[email protected]"}
    fmt.Println(u.Name, u.Email())
}

Name is public, email is private, and Email() is a public accessor -- exactly how you expose read access while keeping a field under the package's control. Within the same package everything is visible; the capital-letter rule only governs what other packages can reach. This single convention replaces a whole vocabulary of access modifiers.

It is worth pausing on how much this simplifies things. In a lot of languages, deciding what is public is a decision you make twice -- once when you write the keyword, and again when you read some documentation to find out what was actually exported. In Go you decide once, at the moment you name the thing, and the name carries that decision everywhere it travels. Rename email to Email and you have made it public; there is nothing else to touch. The rule applies to every top-level name, by the way -- functions, types, constants, variables, and struct fields and methods -- so func parse(...) is a private helper and func Parse(...) is part of your public API, purely by their first letter. Once this clicks you start reading capitalisation as intent: a lowercase name is the author telling you "this is mine, do not rely on it", and a capital one is a promise that it will keep working.

A library package

Any package that is not main is a library -- a reusable unit other packages import. Here is a small one. Notice it has no func main; it just exports some functions and keeps a helper private:

// Package stringutil provides small string helpers.
package stringutil

import "strings"

// Title reports s with its first byte upper-cased. Exported (capital T).
func Title(s string) string {
    if s == "" {
        return s
    }
    return strings.ToUpper(s[:1]) + s[1:]
}

// isVowel is unexported: a private helper, invisible to importers.
func isVowel(r rune) bool {
    return strings.ContainsRune("aeiou", r)
}

// CountVowels is exported and uses the private helper.
func CountVowels(s string) int {
    n := 0
    for _, r := range s {
        if isVowel(r) {
            n++
        }
    }
    return n
}

Title and CountVowels are the package's public surface; isVowel is an implementation detail nobody else can see or depend on. That freedom to change private helpers without breaking anyone is the whole point of the public/private split -- and it is enforced by the compiler, not by convention or documentation.

Notice too that the package name here (stringutil) does not have to match the file name. A package is the whole directory: you could split stringutil across title.go, count.go, and helpers.go, and as long as every file starts with package stringutil they are one package sharing all their names, exported or not. The compiler stitches them together. This is genuinely liberating when a package grows -- you break a long file into several by topic without any ceremony, no re-exporting, no updating an index. The one file name that is special is main.go only by convention; the compiler cares about the package line, not the file's name. And the directory name is what importers type, so a directory called stringutil/ holding package stringutil is the tidy, expected case. Mismatching them (a directory strutil/ holding package stringutil) is legal but confusing, so do not do it without a good reason.

Modules and go.mod

A module is a tree of packages versioned together, rooted at a go.mod file. The module path (usually a repository URL) is also the import prefix for its packages. You create one with go mod init, and go.mod records the module path, the Go version, and any dependencies:

module github.com/scipio/greeter

go 1.27

require github.com/google/uuid v1.6.0

The module line names the module and sets the import prefix; go 1.27 pins the language version; require lists dependencies with exact versions. The commands you will actually use are few: go mod init <path> to start, go get <pkg> to add a dependency, go mod tidy to add missing and drop unused requirements, and go build/go test/go run which resolve everything through go.mod automatically. There is no separate package manager to learn -- it is all go.

Two details are worth knowing early. First, alongside go.mod you will see a go.sum file appear: it records cryptographic checksums of every dependency (and its dependencies) so a build is reproducible and a tampered download is caught. You do not edit it by hand -- go maintains it -- but you do commit it, because it is what guarantees the person who clones your repo tomorrow gets byte-for-byte the same code you built against. Second, the versions in require are real releases pulled from the internet and cached under your module cache (a shared directory on disk), so the second project that needs uuid v1.6.0 does not download it again. Versions follow semantic versioning -- v1.6.0 is major.minor.patch -- and Go leans on that: it will happily pick a newer v1.7.0 for you but treats a jump to v2 as a different import path entirely (the path gains a /v2 suffix), on the principle that a major version bump means "this may break you, so opting in should be explicit". You rarely think about any of this day to day; you type go get, and the machinery keeps the build honest underneath.

Project layout

A real project splits code into packages by responsibility. There is no framework-imposed structure, but a widely used shape puts the runnable entry point under cmd/, private packages under internal/, and importable libraries at the top or under a descriptive directory:

greeter/
  go.mod
  cmd/
    greeter/
      main.go          # package main: the entry point
  internal/
    render/
      render.go        # package render: private to this module
  stringutil/
    stringutil.go      # package stringutil: importable by others

Inside cmd/greeter/main.go, you import your own packages by their full module path, for example import "github.com/scipio/greeter/stringutil" and then call stringutil.Title(...). The import path is the module path plus the directory. Keep packages focused on one responsibility, name them after what they provide (a package is used as stringutil.Title, so stringutil reads well and utils does not), and let the directory tree mirror the design.

Why cmd/ at all? Because a single module often ships more than one binary -- a server and a command-line admin tool, say -- and putting each under cmd/<name>/ gives every entry point its own package main without them colliding, while the reusable logic lives in ordinary packages they both import. Even with a single binary it is a good habit: it keeps main tiny (parse flags, wire things together, call into your real packages) and leaves the actual work in testable library packages. That is the deeper point about naming, too. A package called utils or helpers or common is a warning sign -- it tells you the author had a pile of functions and nowhere to put them, so they invented a junk drawer. Good Go packages are named for a concept (stringutil, render, store) and everything inside relates to that concept. When you find yourself reaching for utils, that is usually the moment to ask what the actual responsibility is and name the package after that. The reward is import sites that read like plain English: render.Page, store.Save, stringutil.Title.

internal: enforced privacy across packages

The public/private rule works within a package, but sometimes you want a whole package that only your own module can import. Go gives you that for free with a directory named internal: a package under internal/ can be imported only by code rooted at internal's parent. Anyone outside the module trying to import it gets a compile error:

github.com/scipio/greeter/
  internal/render/        <- importable ONLY within github.com/scipio/greeter
  stringutil/             <- importable by anyone

So internal/render is usable by your own cmd/greeter and any sibling package in the module, but a different module that tried import "github.com/scipio/greeter/internal/render" would not compile. This is how you keep an implementation package truly private to a project while still splitting it out for organisation -- the boundary is enforced by the toolchain.

The rule is precise and worth stating exactly: a package inside a directory named internal can be imported only by code whose import path shares the parent of that internal directory. So internal/ at the module root is private to the whole module, but you can also nest it -- cmd/greeter/internal/ would be private to just the greeter command. This gives you a lever that the plain capital-letter rule cannot: it lets you export a name (capital letter, so other packages in your module can use it) while still guaranteeing that no outside consumer of your module can reach it. That combination -- public within, sealed without -- is exactly what you want for the guts of a library: you split the implementation across several packages for your own sanity, expose a small clean surface at the top, and the compiler makes sure nobody downstream builds on the parts you meant to keep free to change. I lean on internal/ heavily; it is the difference between "I can refactor this freely" and "somebody, somewhere, imported my plumbing and now I cannot touch it".

Import forms: grouping, aliases, and blank imports

Imports are usually a single grouped block, sorted (which gofmt does for you). Two special forms are worth knowing: an alias renames a package locally (handy for long or clashing names), and a blank import (_) imports a package purely for its side effects -- typically to run its init function, as image formats and database drivers do to register themselves:

package main

import (
    "fmt"
    str "strings"  // alias: refer to strings as str in this file
    _ "image/png"  // blank import: run its init() to register the PNG decoder
)

func main() {
    fmt.Println(str.ToUpper("hello")) // HELLO, via the alias
}

The alias str lets you write str.ToUpper, and the blank _ "image/png" pulls the package in only so its registration init runs -- you never call it directly, so a normal import would be an "imported and not used" error. Aliases are occasional; blank imports show up specifically for these register-yourself side-effect packages.

That "imported and not used" error, by the way, is one of Go's small acts of tidiness: an import you never reference is a compile error, not a warning, so dead imports simply cannot accumulate. The blank import is the sanctioned way to say "yes, I know I am not naming this package, I want it for its side effects only". The classic examples are worth internalising because you will meet them: _ "image/png" and friends register decoders so image.Decode can recognise the format; a database driver like _ "github.com/lib/pq" registers itself with database/sql so sql.Open("postgres", ...) knows what "postgres" means; and _ "net/http/pprof" wires profiling endpoints onto the default HTTP mux. In every case the package's init function does the registering and you never call anything from it by name. Aliases, meanwhile, earn their keep mostly when two imports would otherwise share a name (two different client packages, say) or when a package's name is unwieldy -- reach for them sparingly, because an alias is one more thing a reader has to keep in their head.

init functions and package-level variables

Package-level variables are initialised before main runs. If you need logic to run at startup -- validating config, building a lookup table -- a special init function does it, automatically, after the variables are set and before main:

package main

import "fmt"

var startupMessage string // a package-level variable

// init runs automatically at startup, before main, after vars are initialised.
func init() {
    startupMessage = "ready"
}

func main() {
    fmt.Println(startupMessage) // ready
}

init takes no arguments and returns nothing, you never call it yourself, and a package can even have several. The ordering is well defined and worth knowing: within a package, imported packages are fully initialised first, then package-level variables in dependency order, then any init functions in the order the files are presented to the compiler. That means by the time your init runs, everything it imports is already ready. Use it sparingly all the same -- hidden startup logic can surprise people, because it runs whether or not you meant it to, just by importing the package (that is exactly the mechanism the blank import exploits). The good uses are narrow: registering yourself with some registry, or building a lookup table that is expensive to compute and used everywhere. If setup can be an ordinary constructor the caller invokes explicitly, prefer that -- explicit beats magic.

And a closing word on workspaces: when you are developing several modules together and want them to see each other's local changes without publishing, go work init and a go.work file at the root tie them into one workspace, so a require can resolve to your local copy. It is the multi-module analogue of go.mod, and you reach for it only when juggling more than one module at once -- for example fixing a bug in a library and testing it against the application that uses it, before you have tagged a new release. The go.work file stays on your machine (you usually do not commit it), so it is a development convenience, not part of what you ship.

The same idea in Python

Python organises code into modules (files) and packages (directories), imports with import, and marks "private" only by a leading underscore convention that nothing enforces. Dependencies live in requirements.txt or pyproject.toml, managed by a seperate tool like pip:

# stringutil.py -- a module
import string

def title(s: str) -> str:            # "public" by convention
    return s[:1].upper() + s[1:] if s else s

def _is_vowel(ch: str) -> bool:      # "_private" by convention only (not enforced)
    return ch in "aeiou"

Python leans on convention (_private) and a separate package manager; Go bakes visibility into the language (capital letter = public) and the toolchain (internal/ is enforced, go.mod is built in). The mapping is close enough to be useful when you are moving between the two. Python's module (a .py file) is roughly Go's package (a directory); Python's package (a directory with __init__.py) is roughly Go's module or a subtree of packages; Python's import x and from x import y are Go's import "x" then x.Y; and Python's requirements.txt/pyproject.toml plus pip is Go's go.mod plus the built-in go command. Where they genuinely diverge is enforcement: Python's leading underscore is a polite request that any caller can ignore, and there is no equivalent of internal/ that the interpreter will refuse to cross. Go moved those decisions from "please don't" into "the compiler won't let you", and that is the whole flavour of the language -- take the conventions good engineers already follow, and make the toolchain hold the line so nobody has to remember to. The result is that a Go project's structure is unusually uniform from one codebase to the next -- which is exactly the "easy for a team to read" goal we started the series with.

Exercises

Design the split. Sketch (on paper or as a directory tree) how you would lay out a command-line tool that reads a file, transforms its text, and writes the result: which package holds the transformation logic, which holds the CLI wiring, and what goes under cmd/ and internal/. Justify each placement in a sentence.
Public and private in one package. Write a single package main with an exported type Wallet that has an unexported balance field, an exported NewWallet(initial int) *Wallet constructor, and exported Deposit(int) and Balance() int methods. Show in main that the only way to change the balance is through the methods.
A blank-import reason. Find one standard-library or well-known package that is commonly blank-imported for its side effects (hint: image formats, database/sql drivers, or net/http/pprof), read a paragraph of its docs on go.dev, and write two sentences explaining what its init registers and why you would import it with _.

What we learned

Code lives in packages (one per directory); a name's capital letter makes it exported (public), lowercase makes it package-private -- no access keywords, just case;
A module is a go.mod-rooted tree of packages with a path (the import prefix) and a Go version; manage it with go mod init, go get, go mod tidy, and the ordinary build commands;
Real projects split by responsibility, commonly with the entry point under cmd/, and packages named for what they provide (stringutil.Title, not utils);
A package under internal/ can be imported only within its own module -- privacy enforced across packages by the toolchain, not just convention;
Imports can be aliased (str "strings") or blank (_ "image/png") to run a package's init for side effects; init functions and package-level variables handle one-time startup;
Multi-module workspaces (go.work) let several local modules see each other during development -- the multi-module counterpart to go.mod.

That closes the fundamentals. You can now read and write real Go: types and functions, slices and maps, structs and interfaces, pointers, errors, and the package system that holds a project together. From the next episode we put it to work -- we start building things, beginning with concurrency in earnest and then a run of real programs. Thanks for coming this far.

Tot de volgende keer, en veel plezier met Go! ;-)

@scipio