runtime Package Deep — Optimization¶

1. How to use this file¶

Fourteen scenarios where misuse of the runtime, runtime/debug, runtime/pprof, runtime/trace, and runtime/metrics packages adds latency, contention, STW pauses, or wasted CPU. Each entry has a Before (code + benchmark/measurement) and a collapsible After (optimized code + benchmark + why + trade-offs + when NOT).

Anchored at Go 1.23, amd64, GOMAXPROCS=8. Numbers are reproducible-shape — run go test -bench=. -benchmem on your hardware before quoting them. The runtime package exists for introspection and control, not for steady-state hot paths. The five common failure modes are: (1) calling STW-class functions per request, (2) running profilers at unnecessarily high rates in production, (3) using SetFinalizer as a cleanup mechanism, (4) recomputing global tunables that should be set once at startup, and (5) leaving heavyweight tracing always-on. Most wins remove one of those five from the steady state. Reading order: Ex. 1, 2, 6, then any order. Ex. 4, 9, 12 are the ones most senior reviews flag.

2. Exercise 1 — runtime.ReadMemStats called on every request¶

Difficulty: medium

A /metrics endpoint exposes heap size, GC count, and pause stats. The handler calls runtime.ReadMemStats on every scrape — and the team also added a per-request middleware that exports the same fields. ReadMemStats takes a worldwide stop-the-world pause that scales with the number of P's and the heap. At 8 cores and a 4 GB heap, that pause is ~300 µs per call, multiplied by every in-flight request.

func metricsMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        var m runtime.MemStats
        runtime.ReadMemStats(&m) // STW pause every request
        r = r.WithContext(context.WithValue(r.Context(), "heap", m.HeapAlloc))
        next.ServeHTTP(w, r)
    })
}

BenchmarkReadMemStatsPerReq-8   3500   340000 ns/op   0 B/op   0 allocs/op
// p99 latency under load:  12 ms  (pause-amplified)

import "runtime/metrics"

var (
    samples = []metrics.Sample{
        {Name: "/memory/classes/heap/objects:bytes"},
        {Name: "/gc/cycles/total:gc-cycles"},
    }
    cached atomic.Pointer[snapshot]
)

type snapshot struct{ HeapBytes uint64; GCCycles uint64 }

func init() {
    go func() {
        t := time.NewTicker(10 * time.Second)
        for range t.C {
            metrics.Read(samples)
            cached.Store(&snapshot{
                HeapBytes: samples[0].Value.Uint64(),
                GCCycles:  samples[1].Value.Uint64(),
            })
        }
    }()
}

func metricsMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        if s := cached.Load(); s != nil {
            r = r.WithContext(context.WithValue(r.Context(), "heap", s.HeapBytes))
        }
        next.ServeHTTP(w, r)
    })
}

BenchmarkMetricsCached-8   50000000   24 ns/op   0 B/op   0 allocs/op
// p99 latency under load:  1.2 ms  (no pause)

3. Exercise 2 — runtime.GC() called manually before every benchmark¶

Difficulty: easy

A benchmark suite calls runtime.GC() at the start of every iteration "to get clean numbers". Each forced GC stops the world, sweeps the entire heap, and resets the GC clock. The benchmark now measures GC behavior, not the code under test, and total suite runtime triples.

func BenchmarkParse(b *testing.B) {
    for i := 0; i < b.N; i++ {
        runtime.GC()                  // forces full GC each iter
        _ = parse(input)
    }
}

BenchmarkParse-8   400   2900000 ns/op   ...
// Wall time per iter dominated by GC, not parse

func BenchmarkParse(b *testing.B) {
    b.ReportAllocs()
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        _ = parse(input)
    }
}

BenchmarkParse-8   15000   78000 ns/op   12000 B/op   42 allocs/op

4. Exercise 3 — runtime.SetFinalizer for resource cleanup¶

Difficulty: hard

A file-wrapper sets a finalizer to close the underlying *os.File "in case the caller forgets". Finalizers run on a single goroutine, after at least one GC cycle, with no ordering guarantees relative to program shutdown. Worse, the finalizer keeps the object reachable for an extra GC cycle, doubling its retention time.

type File struct { f *os.File }

func Open(name string) (*File, error) {
    raw, err := os.Open(name)
    if err != nil { return nil, err }
    h := &File{f: raw}
    runtime.SetFinalizer(h, func(h *File) { h.f.Close() }) // fragile
    return h, nil
}

BenchmarkFinalizerOpenClose-8   80000   18500 ns/op   240 B/op   3 allocs/op
// FDs in use after 1e6 opens: ~12000  (finalizer-queue backlog)

type File struct { f *os.File }

func Open(name string) (*File, error) {
    raw, err := os.Open(name)
    if err != nil { return nil, err }
    return &File{f: raw}, nil
}

func (h *File) Close() error { return h.f.Close() }

// caller side:
f, err := Open(name); if err != nil { return err }
defer f.Close()

BenchmarkDeferClose-8   200000   7200 ns/op   80 B/op   1 allocs/op
// FDs in use after 1e6 opens: ~0

5. Exercise 4 — runtime/pprof.StartCPUProfile rate cranked up¶

Difficulty: medium

To "get more samples", a service calls runtime.SetCPUProfileRate(10000) (10 kHz) before starting a continuous CPU profile in production. The default is 100 Hz. At 10 kHz, every P is signaled 10000 times per second, the signal handler walks the stack each time, and CPU overhead jumps from ~1% to ~15%.

func startProfile() {
    f, _ := os.Create("/var/log/cpu.pprof")
    runtime.SetCPUProfileRate(10000)   // 100× default
    pprof.StartCPUProfile(f)
}

// Production measurements:
// CPU overhead from profiler:  ~15%
// p99 latency increase:        +3.2 ms
// Profile file growth:         ~40 MB/min

func startProfile() {
    f, _ := os.Create("/var/log/cpu.pprof")
    // SetCPUProfileRate omitted — defaults to 100 Hz
    pprof.StartCPUProfile(f)
}

// Stop after a 30s window, rotate, restart. Or use net/http/pprof on-demand.

// Production measurements:
// CPU overhead from profiler:  ~0.6%
// p99 latency increase:        +0.05 ms
// Profile file growth:         ~0.4 MB/min

6. Exercise 5 — runtime.SetBlockProfileRate(1) left on in production¶

Difficulty: medium

A service sets runtime.SetBlockProfileRate(1) (sample every blocking event, nanosecond resolution) at startup "for observability". Every channel send/recv that blocks, every mutex contention, every select wait is recorded with a stack. On a chatty service that's millions of records per second; the profile heap inflates and per-op overhead climbs.

func main() {
    runtime.SetBlockProfileRate(1) // every event, forever
    // ...
}

BenchmarkChanSendBlocked-8   2000000   620 ns/op   48 B/op   1 allocs/op
// Block profile size after 1h:  ~800 MB

func main() {
    runtime.SetBlockProfileRate(10000) // 10 µs sampling, ~negligible overhead
    // ...
}

// admin handler for short bursts of high-resolution sampling:
func toggleBlockProfile(w http.ResponseWriter, r *http.Request) {
    rate, _ := strconv.Atoi(r.URL.Query().Get("rate"))
    runtime.SetBlockProfileRate(rate)
    w.WriteHeader(http.StatusOK)
}

BenchmarkChanSendBlocked-8   8000000   155 ns/op   0 B/op   0 allocs/op
// Block profile size after 1h:  ~3 MB

7. Exercise 6 — runtime.Stack(buf, true) polled for monitoring¶

Difficulty: hard

A "stuck-goroutine detector" calls runtime.Stack(buf, true) every 5 seconds to dump all goroutines and grep for ones that haven't moved. runtime.Stack with all=true stops the world and walks every goroutine's stack. At 50k goroutines, that's a 200 ms pause every 5 s — 4% of CPU lost to a monitoring feature.

func monitorStuck() {
    buf := make([]byte, 1<<20)
    for {
        n := runtime.Stack(buf, true) // STW, walks every goroutine
        analyze(buf[:n])
        time.Sleep(5 * time.Second)
    }
}

// At 50k goroutines, 4 GB heap:
// STW pause per call:        ~200 ms
// p99 request latency spike: every 5 s
// CPU spent in runtime.Stack: ~4%

func monitorStuck() {
    t := time.NewTicker(30 * time.Second)
    var lastCount int
    for range t.C {
        n := runtime.NumGoroutine() // ~50 ns, no STW
        if n > 100000 && n > lastCount*2 {
            buf := make([]byte, 1<<20)
            sz := runtime.Stack(buf, true) // one-shot diagnostic dump
            log.Printf("goroutine spike: %d\n%s", n, buf[:sz])
        }
        lastCount = n
    }
}

// At 50k goroutines, 4 GB heap:
// STW pause per call (normal path):  0 ms
// p99 spike from monitoring:         eliminated
// CPU spent in monitor:              <0.01%

8. Exercise 7 — runtime.NumGoroutine() polled in a hot loop¶

Difficulty: easy

A worker pool decides whether to spawn a new goroutine by checking runtime.NumGoroutine() inside the dispatch loop. The call itself is cheap (~50 ns), but it's an atomic load of a per-runtime counter that every P writes to — under load, that cache line ping-pongs across cores.

func dispatch(jobs <-chan Job) {
    for j := range jobs {
        for runtime.NumGoroutine() > 10000 {  // tight poll
            runtime.Gosched()
        }
        go process(j)
    }
}

BenchmarkDispatchPolling-8   500000   2400 ns/op   16 B/op   1 allocs/op
// Cache miss rate on sched.gcount line: ~30%

func dispatch(jobs <-chan Job) {
    sem := make(chan struct{}, 10000)
    for j := range jobs {
        sem <- struct{}{}
        go func(j Job) {
            defer func() { <-sem }()
            process(j)
        }(j)
    }
}

BenchmarkDispatchSemaphore-8   3000000   420 ns/op   16 B/op   1 allocs/op

9. Exercise 8 — runtime.GOMAXPROCS recomputed per request¶

Difficulty: easy

A request handler calls runtime.GOMAXPROCS(runtime.NumCPU()) "to make sure we're using all cores". GOMAXPROCS is a tunable knob, not a query. Calling it with the same value is a no-op fast path; calling it with a different value triggers a full STW to add/remove P's. Either way, it's a hot-path runtime function with no business value.

func handler(w http.ResponseWriter, r *http.Request) {
    runtime.GOMAXPROCS(runtime.NumCPU()) // every request
    serve(w, r)
}

BenchmarkHandlerWithGOMAXPROCS-8   2000000   620 ns/op   0 B/op   0 allocs/op
// vs identical handler without the call:
BenchmarkHandlerClean-8            8000000   145 ns/op   0 B/op   0 allocs/op

import _ "go.uber.org/automaxprocs" // sets GOMAXPROCS from cgroup quota at init

func handler(w http.ResponseWriter, r *http.Request) {
    serve(w, r)
}

BenchmarkHandlerClean-8   8000000   145 ns/op   0 B/op   0 allocs/op

10. Exercise 9 — runtime.LockOSThread for unnecessary "pinning"¶

Difficulty: hard

A goroutine calls runtime.LockOSThread() "for performance — it pins this code to one OS thread". The author was thinking of CPU pinning (sched_setaffinity). LockOSThread does not pin to a CPU; it pins the goroutine to one OS thread, which disables the scheduler's work-stealing for that goroutine. If the goroutine blocks (e.g. on a channel), the whole OS thread parks too. Throughput drops.

func worker(jobs <-chan Job) {
    runtime.LockOSThread()             // not what you think
    defer runtime.UnlockOSThread()
    for j := range jobs {
        process(j)                     // includes channel sends, mutex waits
    }
}

BenchmarkWorkerLocked-8     400000   3100 ns/op   240 B/op   4 allocs/op
BenchmarkWorkerUnlocked-8  1500000    820 ns/op   240 B/op   4 allocs/op

func worker(jobs <-chan Job) {
    for j := range jobs {
        process(j)
    }
}

BenchmarkWorkerUnlocked-8  1500000   820 ns/op   240 B/op   4 allocs/op

11. Exercise 10 — runtime/trace.Start always enabled¶

Difficulty: medium

A service starts runtime/trace.Start at process startup and writes to a 4 GB ring buffer "in case we need to look later". The execution tracer records every goroutine state transition, every GC event, every syscall — at 50k goroutines and 10k req/s, that's ~200 MB/s of trace data, plus per-event overhead inside the runtime.

func main() {
    f, _ := os.Create("/var/log/trace.out")
    trace.Start(f) // always on
    // ...
}

// Production measurements:
// Throughput drop:        ~30%  (10k req/s → 7k req/s)
// CPU overhead:           ~25%
// Disk write rate:        ~200 MB/s

import (
    "net/http"
    _ "net/http/pprof"
)

func main() {
    go func() {
        // admin-only listener, auth-gated
        http.ListenAndServe("127.0.0.1:6060", nil)
    }()
    // ...
}

// Operator runs:  curl -o trace.out 'http://localhost:6060/debug/pprof/trace?seconds=5'

// Production measurements:
// Throughput:                 10k req/s (full)
// CPU overhead (steady):      ~0%
// CPU overhead (during 5s window):  ~25%, bounded

12. Exercise 11 — runtime.Caller(0) in a hot logging path¶

Difficulty: medium

A logger calls runtime.Caller(0) on every log line to attach file:line to the structured event. runtime.Caller walks the stack one frame, then calls runtime.FuncForPC and Func.FileLine, which symbolicate the PC — at >100k logs/sec, this dominates the logger.

func Log(level int, msg string) {
    _, file, line, _ := runtime.Caller(1)
    emit(level, file, line, msg)
}

BenchmarkLogCaller-8   500000   2200 ns/op   144 B/op   2 allocs/op

const filename = "service/handler.go" // or computed once via runtime.Caller in init

func Log(level int, msg string) {
    emit(level, filename, 0, msg)
}

func LogError(level int, msg string) {
    _, file, line, _ := runtime.Caller(1) // pay the cost only for errors
    emit(level, file, line, msg)
}

BenchmarkLogConst-8      30000000   38 ns/op   0 B/op   0 allocs/op
BenchmarkLogErrorRare-8    500000  2150 ns/op  144 B/op   2 allocs/op

13. Exercise 12 — runtime.SetFinalizer to detect leaks in production¶

Difficulty: hard

A team adds runtime.SetFinalizer(obj, func(*T) { atomic.AddInt64(&leaked, 1) }) to every long-lived object to "count leaks". Every finalized object now has to survive an extra GC cycle, finalizer-queue processing is serialized on one goroutine, and the metric only fires after the leak is already gone.

type Conn struct{ /* ... */ }

func NewConn() *Conn {
    c := &Conn{}
    runtime.SetFinalizer(c, func(*Conn) { atomic.AddInt64(&leakedConns, 1) })
    return c
}

BenchmarkNewConnFinalized-8   500000   2900 ns/op   192 B/op   3 allocs/op
BenchmarkNewConnPlain-8      4000000    320 ns/op    96 B/op   1 allocs/op
// Plus: heap retention up ~15% (objects survive extra GC cycle)
// Plus: finalizer goroutine becomes a bottleneck under churn

var (
    connsOpen   atomic.Int64
    connsClosed atomic.Int64
)

func NewConn() *Conn {
    connsOpen.Add(1)
    return &Conn{}
}

func (c *Conn) Close() error {
    connsClosed.Add(1)
    return nil
}

// Expose connsOpen - connsClosed as a gauge; alert on growth without bound.
// Confirm with:  go tool pprof -base old.heap new.heap

BenchmarkNewConnPlain-8   4000000   320 ns/op   96 B/op   1 allocs/op

14. Exercise 13 — Long-lived runtime/pprof files filling the disk¶

Difficulty: medium

A nightly cron starts a pprof.StartCPUProfile writing to /var/log/cpu-YYYYMMDD.pprof and forgets to stop or rotate it. After two weeks, /var/log is full, the process panics on the next write, and journalctl is unreadable. Even without that bug, continuously streaming profile data to local disk burns I/O bandwidth that should be serving requests.

func main() {
    f, _ := os.Create(fmt.Sprintf("/var/log/cpu-%s.pprof", today()))
    pprof.StartCPUProfile(f) // never stopped, never rotated
    // ...
}

// After 14 days:
// Disk used by pprof files: ~40 GB
// Disk I/O baseline:        ~5 MB/s (profile writes)
// Disaster:                 disk full → process panics

func profileLoop(ctx context.Context) {
    t := time.NewTicker(5 * time.Minute)
    defer t.Stop()
    for {
        select {
        case <-ctx.Done():
            return
        case <-t.C:
            captureWindow(30 * time.Second)
        }
    }
}

func captureWindow(d time.Duration) {
    name := fmt.Sprintf("/var/log/pprof/cpu-%d.pprof", time.Now().Unix())
    f, err := os.Create(name)
    if err != nil { return }
    defer f.Close()
    if err := pprof.StartCPUProfile(f); err != nil { return }
    time.Sleep(d)
    pprof.StopCPUProfile()
    // upload f to object storage, then delete locally
    uploadAndPrune(name)
}

// After 14 days:
// Local disk used by pprof: bounded by retention policy (~500 MB)
// I/O baseline:              ~negligible (uploads run async)
// No more disk-full panics.

15. Exercise 14 — runtime/debug.FreeOSMemory() called periodically¶

Difficulty: medium

A team adds debug.FreeOSMemory() on a 60 s timer "to give memory back to the OS faster". FreeOSMemory forces a full GC cycle and a synchronous scavenger pass that walks the heap and madvise(MADV_DONTNEED)s free pages. At a 4 GB heap, that's a ~400 ms STW + scavenger cost every 60 s — a noticeable latency spike every minute.

func init() {
    go func() {
        t := time.NewTicker(60 * time.Second)
        for range t.C {
            debug.FreeOSMemory() // STW + scavenger every minute
        }
    }()
}

// Production measurements (4 GB heap):
// Every 60s:  ~400 ms p99 latency spike
// RSS:        marginally smaller (~5%) than without

// In your deployment manifest / systemd unit:
// GOMEMLIMIT=3GiB
// GOGC=80   (optional: trade CPU for lower heap)

// Code: nothing.

// Production measurements (4 GB heap):
// p99 latency spikes:    none (background scavenger is async)
// RSS:                   ~same steady-state ceiling, smoother curve

16. When NOT to optimize¶

A handful of the patterns above are correct for some code:

• runtime.GC() belongs in microbenchmarks of GC itself, in tests that need a clean heap to assert on object liveness, and exactly nowhere else.
• runtime.SetFinalizer is correct for cgo handles whose lifetime can't be expressed with defer Close() — and even there, an explicit Close is preferred when possible.
• runtime.LockOSThread is required for OpenGL, Cocoa main thread on macOS, some CGo libraries with thread-local state, and Linux namespace operations like setns and unshare.
• High-rate pprof (SetCPUProfileRate > 100) makes sense in microbenchmarks of microsecond-scale functions.
• runtime.Stack(_, true) is the right tool for SIGQUIT crash dumps and forensic post-mortems.
• runtime/trace should be on full-time in pre-production load tests where you're hunting scheduler pathologies.
• runtime/pprof files written to disk are fine for one-shot diagnoses on a developer machine.
• debug.FreeOSMemory is reasonable at coarse phase boundaries in long-running batch jobs.

The common thread: the runtime package is for introspection and tuning, not for per-request work. Anything that takes a stop-the-world pause, walks the entire heap, or pays symbolication cost belongs out of the hot path. Anything that has a "set once at startup" feel — GOMAXPROCS, SetBlockProfileRate, SetMutexProfileFraction, SetMemoryLimit — should be set once in main (or via env var) and never touched again. Anything that records data continuously — trace.Start, high-rate profiles — should run for a bounded window, on demand.

When in doubt: measure before you reach for runtime.*. The Go runtime is already heavily tuned for typical workloads. The default GOGC=100, default GOMAXPROCS=NumCPU, default 100 Hz CPU profiling, default 0 (off) block/mutex profiling, default async background scavenger — these defaults are right for ~95% of production code. Override them only when a profile shows the default is measurably wrong for your workload, and document why in a comment next to the call.

17. Summary¶

The runtime package is your introspection scope. It is not a knob to be turned per request. Set once, observe sparingly, override defaults only when the profile demands it.