Memory Profiling in Go — Specification¶

Focus: Precise reference for capturing, sampling, and interpreting Go heap profiles — the APIs in runtime/pprof and net/http/pprof, the four profile metrics, the sampling math behind runtime.MemProfileRate, and the go tool pprof commands that read the output.

Sources: - runtime/pprof package: https://pkg.go.dev/runtime/pprof - net/http/pprof package: https://pkg.go.dev/net/http/pprof - runtime.MemProfileRate: https://pkg.go.dev/runtime#MemProfileRate - go tool pprof docs: https://github.com/google/pprof/blob/main/doc/README.md - Profiling Go programs: https://go.dev/blog/pprof

1. What a memory profile actually contains¶

A Go heap profile is a list of sampled allocation records. Each record carries:

Each record represents the scaled-up estimate inferred from the samples actually recorded. The profile is statistical, not exact — see §5.

2. The four metrics, paired¶

A site that allocates a million 16-byte structs and a site that allocates ten 1 MiB buffers may have the same alloc_space but very different alloc_objects. The GC cost is roughly per-object, so alloc_objects is the better signal for GC CPU; alloc_space is the better signal for heap pressure.

3. Capture APIs in runtime/pprof¶

debug=0 writes the binary protobuf consumed by go tool pprof. debug=1 writes a human-readable text dump. debug=2 writes a goroutine dump in the same format as a panic.

import (
    "os"
    "runtime"
    "runtime/pprof"
)

f, _ := os.Create("heap.pb.gz")
defer f.Close()
runtime.GC()                     // get a clean live set first
pprof.WriteHeapProfile(f)        // writes inuse_* metrics

The leading runtime.GC() is a deliberate trade: it makes inuse_* reflect only what survived a full collection, eliminating "garbage we haven't swept yet" from the profile.

4. The HTTP endpoint surface¶

Importing net/http/pprof registers handlers on http.DefaultServeMux:

import _ "net/http/pprof"
import "net/http"

go http.ListenAndServe("127.0.0.1:6060", nil)

Then:

go tool pprof http://localhost:6060/debug/pprof/heap
go tool pprof http://localhost:6060/debug/pprof/allocs

5. Sampling: runtime.MemProfileRate¶

The runtime decides whether to record an allocation by drawing from a geometric distribution with mean MemProfileRate. So a 64-byte object has about a 64/524288 ≈ 0.012% chance of being recorded, while a 1 MiB object will almost certainly be recorded.

import "runtime"

func init() {
    runtime.MemProfileRate = 1   // tests only
}

The runtime scales recorded samples back up by the sampling rate when generating the profile. The output bytes look "correct"; the per-site numbers are estimates with variance proportional to 1/√samples.

6. go test profiling flags¶

go test -bench=. -benchmem -memprofile=mem.out -memprofilerate=1 ./pkg
go tool pprof mem.out

-benchmem is a per-benchmark report computed from runtime.MemStats deltas, not from sampled profiles. Its numbers are exact for that benchmark, regardless of MemProfileRate.

7. go tool pprof commands you actually use¶

Once inside the interactive shell:

Out of the shell:

go tool pprof -http=:8080 heap.pb.gz                 # web UI with flame graph
go tool pprof -top -sample_index=alloc_objects heap.pb.gz
go tool pprof -base old.pb.gz new.pb.gz              # diff profile

The web UI (-http) is the modern default — it ships a flame graph, a callgraph, a top list, and a source view in one page.

8. Flame graphs¶

A wide frame near the top is "this leaf allocates a lot directly". A wide frame near the bottom that narrows quickly is "lots of paths converge into one entry point". Use granularity=lines for line-level resolution.

9. Diff profiles¶

go tool pprof -base before.pb.gz after.pb.gz
(pprof) top

The top view now shows delta allocations: after - before. Positive values are sites that allocated more after the baseline; negative values, less. Useful for:

curl -o t0.pb.gz http://prod:6060/debug/pprof/heap
sleep 1800
curl -o t1.pb.gz http://prod:6060/debug/pprof/heap
go tool pprof -base t0.pb.gz -http=:8080 t1.pb.gz

10. Relationship to escape analysis¶

A profile tells you where allocations happened. -gcflags=-m tells you why a particular variable escaped. They're complementary:

Typical workflow: profile picks the hot site; escape report explains why it's hot; you change the code; profile confirms the change.

11. Continuous heap profiling¶

The shared idea: scrape sampled profiles at a low cadence, store them, let an operator query "what changed between this hour and last hour" without ever logging into a box. Overhead is negligible because the default MemProfileRate=512KiB is already cheap.

12. Non-goals and limits¶

• The heap profile does not show stack allocations — escape analysis decides that, and stack allocations are invisible to pprof.
• The heap profile does not show C allocations (malloc via cgo) — those are below the Go runtime and require a separate tool (jemalloc's MALLOC_CONF=prof:true, heaptrack, valgrind).
• inuse_* is a snapshot at profile time; it cannot reconstruct who held a reference. Use goroutine profile or a heap dump tool for that.
• Sample bias: very small allocations under MemProfileRate may not appear at all in any single profile.

13. Related references¶

• The pprof README: https://github.com/google/pprof/blob/main/doc/README.md
• runtime/pprof source: https://github.com/golang/go/tree/master/src/runtime/pprof
• Flame graphs (Brendan Gregg): https://www.brendangregg.com/flamegraphs.html
• Profiling Go programs (Go blog): https://go.dev/blog/pprof
• Memory profiling internals (Felix Geisendörfer): https://github.com/DataDog/go-profiler-notes/blob/main/heap.md