CPU Profiling in Go — Specification¶

Focus: Precise reference for how Go captures CPU profiles — the sampling mechanism, the public APIs, the wire format, and the tools that read them.

Sources: - runtime/pprof package: https://pkg.go.dev/runtime/pprof - net/http/pprof package: https://pkg.go.dev/net/http/pprof - testing package profiling flags: https://pkg.go.dev/testing - pprof tool: https://github.com/google/pprof - pprof profile format: https://github.com/google/pprof/blob/main/proto/profile.proto

1. What a CPU profile is¶

A CPU profile is a statistical sample of program execution. The runtime periodically interrupts the program and records the call stack of whichever goroutine was running on each thread. The resulting set of stacks is aggregated into a weighted call graph: each frame's weight is proportional to the number of samples in which it appeared.

A 30-second profile at 100 Hz on 8 cores yields up to 24,000 samples — enough resolution to find any function consuming ≥0.5% of CPU.

2. How sampling works under the hood¶

kernel → SIGPROF → signal handler → record stack of current goroutine → continue

1. The runtime calls setitimer(ITIMER_PROF, ...) to request a periodic signal.
2. On each tick, the kernel delivers SIGPROF to one thread at a time (round-robin).
3. The signal handler walks the goroutine's stack using compiler-emitted metadata and writes an entry to the lock-free profile buffer.
4. A reader goroutine drains the buffer and writes a pprof profile to the output writer.

Because the signal is delivered per-thread, the sampling rate scales with GOMAXPROCS: at 100 Hz with 8 threads, expect ~800 samples/second across all cores.

3. Public APIs¶

runtime/pprof¶

Only one CPU profile may be active at a time. A second StartCPUProfile returns an error.

runtime¶

SetCPUProfileRate(0) disables profiling. Rates above 1000 Hz are clamped on most platforms because the kernel will not deliver SIGPROF faster than that.

net/http/pprof¶

Importing _ "net/http/pprof" registers these handlers on http.DefaultServeMux:

testing¶

These are honored by the standard testing.M flow without any code changes.

4. Profile file format¶

The on-disk format is a gzipped profile.proto (https://github.com/google/pprof/blob/main/proto/profile.proto).

Profile {
  repeated SampleType sample_type;  // e.g., "samples/count", "cpu/nanoseconds"
  repeated Sample sample;           // each sample: location stack + values
  repeated Location location;       // PC -> function + line
  repeated Function function;
  repeated Mapping mapping;         // module mappings
  repeated string string_table;
  int64 time_nanos;
  int64 duration_nanos;
  int64 period;                     // sampling period in nanoseconds (e.g., 10000000 for 100 Hz)
}

Two sample values per sample for a CPU profile: count (number of samples that hit this stack) and CPU nanoseconds (count × period). Tools display either; default is CPU time.

5. The go tool pprof interface¶

go tool pprof [flags] [binary] profile.pprof

Flags:

The web UI (-http) is the modern default; the interactive REPL remains useful in CI and ssh sessions.

6. Flat vs cumulative time¶

A leaf function (e.g., runtime.memmove) has flat ≈ cum. A wrapper function (http.serverHandler.ServeHTTP) has tiny flat but huge cum.

The first question to ask of any profile: "What has high flat?" Those are real hotspots. Functions with only high cum are not where the work happens — they are where it is organized.

7. Runtime functions you will see¶

Seeing many seconds in runtime.findrunnable is idle CPU, not work — the profile counts on-CPU time for whatever the kernel scheduled. Filter with -ignore=runtime\\.findrunnable.

8. Profile labels (pprof.Do)¶

pprof.Do(ctx, pprof.Labels("endpoint", "/api/v1/users"), func(ctx context.Context) {
    handleUsers(ctx)
})

Samples taken while the goroutine carries these labels record them. go tool pprof can then group:

(pprof) tags endpoint
(pprof) -tagfocus=endpoint=/api/v1/users

Labels are propagated across go statements that capture the labeled context. Cost: a few words per Do call. Use to attribute CPU to logical units (request types, tenants, batch jobs).

9. Sample bias and limitations¶

A CPU profile answers "where did CPU go", not "why was the request slow". For latency questions, use traces (go tool trace) plus block/mutex profiles.

10. Non-goals¶

• pprof CPU profiling is not a code coverage tool. Cold paths are absent from the profile.
• It is not a wall-clock latency profile. Off-CPU time (sleeping on I/O, locks) is invisible.
• It cannot attribute time inside C libraries beyond the entry point.
• It cannot tell you why memory was allocated — that is the heap profile's job.

11. Related references¶

• pprof README: https://github.com/google/pprof/blob/main/doc/README.md
• "Profiling Go Programs" (Go blog, classic): https://go.dev/blog/pprof
• runtime/pprof source: https://github.com/golang/go/tree/master/src/runtime/pprof
• Continuous profiling overview: https://grafana.com/oss/pyroscope/