When to Use a Pool — Senior Level¶

Table of Contents¶

1. Introduction
2. Deep Comparison: API Ergonomics
3. Deep Comparison: Locking Strategies
4. Deep Comparison: Allocations Per Task
5. Deep Comparison: Scheduling Fairness
6. Deep Comparison: Memory Footprint
7. Microbenchmarks With Real Numbers
8. Profile-Driven Choices
9. Cache Behaviour and False Sharing
10. GC Interactions
11. Scheduler Interactions
12. Tail Latency Analysis
13. Failure Modes and Recovery
14. Race-Detector Behaviour
15. Senior Decision Process
16. Real-World Architecture Case Studies
17. Anti-Patterns at Senior Level
18. Designing for the Long Run
19. Self-Assessment
20. Summary

Introduction¶

Focus: "I have built and operated services that use pools. I now want to know why one pool wins over another, down to the cache line."

The senior level of this subsection is about why. You already know which tool to reach for in 90% of cases; this file is for the other 10% and for the moments when you need to explain a benchmark result, dig into a profile, or argue technical tradeoffs in a design review.

We will:

• Compare the internals of errgroup, semaphore.Weighted, ants, tunny, workerpool, and pond at the level of "what locks does each acquire on submit?"
• Show pprof snippets from real benchmarks.
• Discuss allocations per task — including the difference between a stack-allocated closure and a heap-allocated one.
• Examine scheduling fairness — what each tool guarantees about task ordering and worker selection.
• Analyse memory footprint at idle and at peak — what's the bytes-per-worker, bytes-per-pending-task.
• Examine cache behaviour and false sharing in pool implementations.
• Discuss GC interactions: how pool task closures affect heap pressure.
• Look at scheduler interactions: how pools play with GOMAXPROCS, work-stealing, and preemption.
• Walk through tail latency analysis: why p99 spikes, and how to debug them.

This file expects you to have shipped pool-using code, read pprof output, and at least skimmed the scheduler's design doc. If not, retreat to middle.md.

Deep Comparison: API Ergonomics¶

API ergonomics is not a soft topic — it is the thing that determines how often your team gets concurrency right. A library with terse, well-named APIs reduces bugs; one with verbose, ambiguous APIs causes them.

We will compare the libraries along five concrete dimensions.

Dimension 1: Type safety of the Submit signature¶

The Submit signature shapes how you reason about the task.

The clear winner on ergonomics: errgroup. The clear loser: tunny (because of any).

But ergonomics is not the only factor. tunny's any makes sense when the worker state and the task payload are fundamentally typed at runtime (e.g., a polymorphic worker). For most uses, this is a downside.

Dimension 2: Submitting multiple tasks¶

How does the API support "submit N tasks and wait for all"?

errgroup and pond have first-class "submit-and-wait" patterns; others require manual WaitGroup wiring.

Dimension 3: Result collection¶

How do you get results back from N tasks?

tunny is unique in returning a typed value from each call. For workloads with results, tunny's API is the most natural.

Dimension 4: Cancellation¶

How does the API integrate with context.Context?

errgroup and semaphore have first-class ctx. Pools do not. This is one of the biggest reasons errgroup is the default for fan-out.

Dimension 5: Construction / teardown¶

How heavy is the lifecycle?

errgroup and semaphore have the lightest lifecycle (per-call construction). Pools require explicit construction and teardown — which is fine for long-lived pools, awkward for per-call usage.

Verdict on ergonomics¶

Across these five dimensions:

• errgroup wins on type safety, multi-task, cancellation, lifecycle.
• tunny wins on result collection.
• ants is middle on most; loses on type safety, ctx.
• pond is competitive; richer than ants in some places.
• workerpool is the minimal API; few features but very small surface.

If ergonomics is the deciding factor, prefer errgroup. If you need ergonomics and a feature (result type, worker reuse), the choice depends on the feature.

Deep Comparison: Locking Strategies¶

Inside each library, how does Submit work? What locks are acquired? What is contended?

errgroup¶

errgroup.Group with SetLimit(K) uses an internal chan struct{} of capacity K as a semaphore. g.Go does:

1. Send into the channel (blocks if K already in flight).
2. Spawn the goroutine.
3. The goroutine, when done, receives from the channel.

The lock is the channel's internal mutex. Under heavy contention, the channel can serialise senders. Bench shows ~150 ns per Go call below the limit; ~2-3 μs when at the limit (including spawn).

semaphore¶

semaphore.Weighted uses a sync.Mutex protecting an internal counter and a wait-list. Acquire(ctx, n):

1. Lock the mutex.
2. If n <= free, decrement free, unlock, return.
3. Else add a waiter to the list, unlock, wait on a per-waiter channel.
4. Release(n) locks, increments free, wakes the head of the wait-list.

Under contention, the mutex is the bottleneck. ~100 ns Acquire when free; longer when blocked.

ants¶

ants pool uses a custom worker queue and a sync.Mutex protecting the worker slice. Submit:

1. Try to fetch an idle worker (lock, pop from stack, unlock).
2. If no idle and pool not full: spawn a new worker.
3. If pool full and blocking: wait on a condition variable.
4. If non-blocking: return ErrPoolOverload.

ants uses a stack of idle workers (LIFO) — newer-idle workers serve next, which is good for cache locality. Submit is ~250 ns under low contention; the lock is a hotspot above ~10k submits/sec from multiple goroutines.

Recent ants versions added "loop queue" mode (WithLoopQueue) that reduces lock contention by using a circular buffer with atomic operations. Submit drops to ~150 ns under contention with this option.

tunny¶

tunny has one Goroutine per Worker, each owning a dispatch channel. The pool has a "request channel" that fans out to whatever worker is free.

Submit (Process):

1. Send the payload to the pool's input channel (blocks if all workers are busy).
2. The dispatcher routes to an idle worker's channel.
3. The worker processes, sends back via a response channel.

There is no mutex — all coordination is via channels. The serialisation point is the dispatch loop. Process is ~600 ns at low contention; gets worse at high contention because dispatch is single-threaded.

workerpool¶

workerpool uses a single chan func() task channel feeding K workers. Submit:

1. If a worker is idle and channel is empty, send directly.
2. Else queue in a slice (with a mutex).
3. Dispatcher feeds from queue to channel.

The mutex on the queue is a hotspot at high contention. Submit is ~300 ns nominally; up to 1-2 μs at high contention.

pond¶

pond is similar to ants — a custom queue with sharding. Submit acquires a lock on one of N internal queues (sharded by some hash). Lower contention than a single-lock design.

Summary table¶

These numbers are illustrative; rerun on your hardware. The shape — which one is fastest under contention — is stable across hardware.

What this means¶

At low throughput (<1k submits/sec from few producers), the difference is invisible. At very high throughput (>100k submits/sec from many producers), the difference is 5-10x. If your benchmark shows a tight loop dominated by submit overhead, the library choice matters.

Deep Comparison: Allocations Per Task¶

A pool's "allocations per task" affects both CPU (allocations cost) and GC pressure (heap survivor work).

Where do allocations come from?¶

1. The task closure. pool.Submit(func() { ... }) allocates a closure on the heap (unless the compiler can prove it's stack-allocatable, which is rare for closures captured in a for loop).
2. The task queue node. Some libraries wrap the closure in a queue-element struct.
3. Per-worker state at construction. One-time, doesn't show up per-task.

Allocations per task: real numbers¶

Measured via go test -benchmem:

ants has the surprising result of 0 allocs/op when the closure captures nothing — because the closure can be a function pointer rather than a heap-allocated function value. When the closure captures variables, you get one alloc per capture.

errgroup has higher per-call allocation because it builds an internal sync structure each time.

For workloads with billions of tasks (rare), the alloc/op number matters. For typical workloads, the difference is invisible.

Reducing allocations¶

Several techniques:

1. Avoid closures with captures. Use tunny-style payload passing where the typed payload travels through the system. The task function itself has no captures.

2. Reuse closures. If you're submitting the same function for many items, define one closure with state in a struct:

type Submitter struct {
    pool *ants.Pool
}

func (s *Submitter) Run(items []Item) {
    for _, it := range items {
        s.runOne(it)
    }
}

func (s *Submitter) runOne(it Item) {
    s.pool.Submit(func() { processItem(it) })
}

The closure captures only it, which is a simple value. The compiler may stack-allocate if it is a small value type — but for an interface or pointer, it allocates.

1. Use typed pool variants. ants.MultiPool and ants.PoolWithFunc[T] (with one stored function) avoid per-task closure allocation. The function is set once; payloads are passed:

pool, _ := ants.NewPoolWithFunc(K, func(arg any) {
    // arg is the payload
    work(arg.(Item))
})

pool.Invoke(Item{...}) // no closure!

Invoke is faster than Submit because there's no closure to allocate. But the worker's function is fixed at pool construction.

1. Batch. Instead of submitting 1000 tiny tasks, submit 10 "batch" tasks each processing 100 items. The closure overhead is paid once per batch, not per item.

When alloc/op matters¶

• Million-tasks-per-second workloads (game servers, ad-tech).
• Memory-constrained environments where GC pressure is the bottleneck.
• Real-time systems where allocation pauses are problematic.

For typical CRUD services at moderate RPS, alloc/op is noise.

When alloc/op doesn't matter¶

Most of the time. If each task takes 1 ms of work, a 100 ns allocation is 0.01% of the work. Optimise elsewhere.

Deep Comparison: Scheduling Fairness¶

When you submit N tasks to a pool, in what order do they execute? What does each library guarantee?

Pool ordering semantics¶

Why LIFO in ants? When the workload has bursts of activity then idleness, the newest-idle worker has the warmest cache. LIFO maximises cache hits. The cost is older workers may sit idle and eventually expire (which may be desirable).

FIFO benefit: predictability. If task A arrives before task B, A starts before B (modulo brief reordering). Important for fairness across tenants or task classes.

When fairness matters¶

• Multi-tenant systems: if tenant A submits 1000 tasks and tenant B submits 1 task, you want tenant B's single task to start before all 1000 of A's. LIFO breaks this; FIFO partially solves it (tenant B's task is at the back of the FIFO queue but doesn't wait for all 1000 of A's to complete — only until a worker is free). The right tool here is per-tenant queues — neither LIFO nor FIFO of a single queue.

• Latency-sensitive interleaved workloads: tail latency depends on worst-case wait. FIFO gives predictable wait; LIFO can starve.

• Backpressure visibility: FIFO with a bounded queue gives clear "queue depth = wait time" intuition. LIFO does not.

How to enforce fairness¶

If you need strict fairness:

• Use FIFO pools (workerpool, tunny, ants-loopqueue).
• Avoid LIFO defaults of ants.
• Add per-tenant queues or token buckets.

If you need cache-friendly LIFO:

• Use ants default.
• Accept that under sustained load, the same workers serve repeatedly.

A worked example: starvation under LIFO¶

Imagine a pool of 4 ants workers. A producer submits 1 task. Worker 1 takes it. Idle workers 2, 3, 4 are placed on the stack in some order. Worker 1 finishes; the next task arrives — Worker 1 is on top of the stack, takes it. Workers 2, 3, 4 remain idle. With low traffic, Workers 2, 3, 4 may sit idle indefinitely until they expire.

This is usually fine: under low load, you don't need 4 workers anyway. But if Workers 2, 3, 4 are expensive to construct, you wanted them warm. Switch to FIFO (WithLoopQueue) or to tunny which uses a different model.

Deep Comparison: Memory Footprint¶

How much memory does each library consume?

Per-pool baseline¶

The pool object itself (not workers, not tasks): typically 100-500 bytes. Negligible.

Per worker¶

A worker is a goroutine. Initial stack: ~2 KB. With pool-specific state, perhaps another 200-1000 bytes.

For a pool of 1000 workers: ~3 MB. Negligible on modern hardware.

Per pending task¶

For 1M pending tasks in a queue: 50 MB. Real, but only at extreme queue depths.

Total at peak¶

For a typical service: K=500 pool, queue depth=2000 max.

Workers: 500 × 3 KB = 1.5 MB. Pending tasks: 2000 × 50 bytes = 100 KB.

Total pool memory: ~1.6 MB. Trivial.

For a massive service: K=10000, queue depth=100000.

Workers: 10000 × 3 KB = 30 MB. Pending tasks: 100000 × 50 bytes = 5 MB.

Total: 35 MB. Still trivial relative to typical service heaps (hundreds of MB to GB).

When memory matters¶

• Embedded / resource-constrained environments (IoT, edge): a 30 MB pool may be 10% of available RAM.
• Containers with tight limits: the pool's overhead competes with your hot data.
• Very large pools at idle: if you size K to peak but the system runs at 10% peak most of the time, you're paying for 10× more workers than needed. Tune lower or use idle expiry.

Idle-time memory¶

If workers expire after idle, the pool shrinks. ants default: 1 second idle = expire. The pool returns to a minimal size between bursts.

If you want workers to stay warm: disable expiry (set very high) or use tunny (no expiry).

Stack growth¶

Go goroutines grow their stack on demand. A pool worker that runs the same shallow function will keep a small stack. A worker that occasionally runs a deeply recursive function will grow its stack and the stack will stay larger for the worker's life. This is mostly invisible but can surprise you at peak.

Microbenchmarks With Real Numbers¶

Below are illustrative go test -bench results from a typical M1 Pro laptop. Numbers will differ on Linux x86 by 1.5-3x.

Benchmark: 100k atomic-increment tasks¶

func BenchmarkRaw(b *testing.B) {
    var counter atomic.Int64
    for i := 0; i < b.N; i++ {
        var wg sync.WaitGroup
        for j := 0; j < 100000; j++ {
            wg.Add(1)
            go func() { defer wg.Done(); counter.Add(1) }()
        }
        wg.Wait()
    }
}

func BenchmarkErrgroup(b *testing.B) {
    var counter atomic.Int64
    for i := 0; i < b.N; i++ {
        g, _ := errgroup.WithContext(context.Background())
        g.SetLimit(1024)
        for j := 0; j < 100000; j++ {
            g.Go(func() error { counter.Add(1); return nil })
        }
        _ = g.Wait()
    }
}

func BenchmarkAnts(b *testing.B) {
    var counter atomic.Int64
    pool, _ := ants.NewPool(1024)
    defer pool.Release()
    for i := 0; i < b.N; i++ {
        var wg sync.WaitGroup
        for j := 0; j < 100000; j++ {
            wg.Add(1)
            pool.Submit(func() { defer wg.Done(); counter.Add(1) })
        }
        wg.Wait()
    }
}

Results (approximate):

BenchmarkRaw-10           5    250ms/op    200000 allocs/op
BenchmarkErrgroup-10     10    180ms/op    300000 allocs/op
BenchmarkAnts-10         20     85ms/op         5 allocs/op

ants is ~3x faster on this workload, with 40,000x fewer allocations. The work is so cheap (atomic.Add) that overhead dominates.

Benchmark: 100k 100-μs CPU tasks¶

Each task does a 100 μs CPU burst (some loop).

BenchmarkRaw-10          50      30ms/op    200000 allocs/op
BenchmarkErrgroup-10     50      32ms/op    300000 allocs/op
BenchmarkAnts-10         50      29ms/op         5 allocs/op

When the work itself is 100 μs, overhead is ~0.3% of the runtime. All three approaches are within noise.

Benchmark: 1k HTTP-call tasks¶

Each task makes one HTTP GET to a local server (median ~5 ms).

BenchmarkRaw (unbounded)-10      1   1.1s/op   2000 allocs/op (peak 1 GB memory)
BenchmarkErrgroup K=100-10       1   2.0s/op   3000 allocs/op (peak 50 MB)
BenchmarkAnts K=100-10            1   2.0s/op   1000 allocs/op (peak 50 MB)

At I/O-heavy workloads with bounding, all bounded approaches are equivalent. The "raw unbounded" version is faster but uses 20× memory.

Benchmark: tunny vs errgroup with warm-state init¶

Each task uses a 50ms warm state (regex compile). With tunny, the regex is compiled once per worker; with errgroup, compiled per task.

BenchmarkErrgroup K=4-10       1     12.6s/op    250 allocs/op
BenchmarkTunny K=4-10          1      3.5s/op     50 allocs/op

tunny wins by 4× on this workload because of warm-state amortisation.

Reading benchmark results¶

Three takeaways:

1. At low task cost, pool overhead matters; ants wins.
2. At high task cost (real I/O, real CPU), overhead is invisible; any bounded approach is fine.
3. At warm-state workloads, tunny wins by reusing the state.

Benchmark your actual workload. Generic benchmarks (atomic.Add, sleep, no-op) don't predict your service's behaviour.

Profile-Driven Choices¶

When choosing or tuning a pool, profile your service. Here is how to extract the right data.

CPU profile¶

import _ "net/http/pprof"

go func() {
    http.ListenAndServe(":6060", nil)
}()

Then go tool pprof -seconds=30 http://localhost:6060/debug/pprof/profile.

In the profile, look for:

• Time in runtime.newproc or runtime.goexit0: goroutine creation. High percentage means you're paying for spawn — consider a pool.
• Time in sync.Mutex or chan send: pool internal lock or channel contention. Check your tool choice.
• Time in pool-specific code (e.g., ants.(*Pool).Submit): how much of CPU is pool dispatch.

If pool dispatch is >5% of CPU, the pool is too small for the workload (saturated) or the wrong shape (lock-heavy).

Heap profile¶

go tool pprof http://localhost:6060/debug/pprof/heap

Look for:

• Goroutine stacks: if runtime.gohacks or similar shows many MB, you have too many goroutines.
• Task closures: look for *func() and *funcInfo types. Many of these = high task throughput. Consider Invoke-style or batching.
• Worker state: per-worker structs.

Goroutine profile¶

go tool pprof http://localhost:6060/debug/pprof/goroutine

Look for:

• Total goroutine count: should be steady. Growing = leak.
• Goroutines blocked on chan send: producers blocked because pool is saturated.
• Goroutines parked in pool worker loops: idle workers. Their count should be ≤ pool cap.

Block profile¶

go tool pprof http://localhost:6060/debug/pprof/block

Block profile shows where goroutines sit waiting. For a pool, you want to see:

• Workers blocking on the task channel (idle, fine).
• Producers blocking on Submit (saturated; expected under heavy load).

If you see workers blocking on a downstream lock or channel, that's the bottleneck — not the pool.

Mutex profile¶

go tool pprof http://localhost:6060/debug/pprof/mutex

Mutex profile shows contended locks. If the pool's internal mutex is high here, consider switching to ants-loopqueue or pond (sharded). If it's your mutex inside a task, the pool isn't the problem.

Tail latency¶

# expvar or prometheus-style histogram

Plot p50, p95, p99 over time. Spikes in p99 with stable p50 indicate queue buildup at peak. Investigate: is the pool saturated? Is the queue unbounded? Is a downstream slow?

Cache Behaviour and False Sharing¶

A senior topic that comes up in extreme-perf pool work.

What is false sharing?¶

When two CPU cores write to different variables that happen to live on the same cache line (typically 64 bytes), each write invalidates the other's cache, causing slow synchronisation. The variables are "falsely shared" — they don't actually share data, but they share a cache line.

In a pool, this can happen when:

• Multiple workers update adjacent counters in a slice (e.g., per-worker stats packed too tightly).
• The pool's internal queue head/tail pointers live on the same line.

How libraries avoid it¶

• ants: uses cache-line padding in some internal structs (since v2.5 or so).
• tunny: less concern, because per-worker state is separately heap-allocated.
• workerpool: typically not optimised; for typical workloads, fine.
• pond: sharded queues, naturally less false sharing.

When to care¶

False sharing matters at >10M ops/sec from multiple cores. For typical pool workloads, invisible. If a profile shows surprising CPU on memory traffic, investigate.

How to fix it¶

Cache-line padding:

type PaddedCounter struct {
    v int64
    _ [56]byte // pad to 64 bytes
}

For workloads where you measured a false-sharing problem.

GC Interactions¶

Pools affect the garbage collector in subtle ways.

Task closures and the GC¶

Each task closure (when allocated on the heap) is a small allocation that the GC must trace. Millions of closures per second = significant GC pressure.

Symptoms of pool-induced GC pressure:

• p99 latency spikes during GC.
• runtime.gcBgMarkWorker high in CPU profile.
• go_memstats_gc_duration_seconds (Prometheus) high.

Mitigations:

1. Reduce per-task allocations (batch, Invoke-style, no captures).
2. Tune GOGC higher (e.g., GOGC=200) to GC less often. Trades memory for less CPU on GC.
3. Use object pools (sync.Pool) for task payload types.

Pool workers and GC¶

Pool workers are long-lived goroutines. Their stacks may grow over time. The GC traces all stacks, so a pool of 10,000 workers with 8 KB stacks each = 80 MB of stack to trace per GC cycle.

For most services, fine. For services where GC pause matters, monitor.

Tunny and worker state¶

If each tunny worker holds a large cache (e.g., 100 MB regex), the total heap is K × 100 MB. The GC traces all of this. Tunny workers with big state can produce big GC.

errgroup and GC¶

errgroup creates a goroutine per task. Goroutine creation allocates a goroutine struct and stack. GC sees these. Errgroup-heavy workloads can have higher GC than pool-based workloads, in absolute numbers — though GC is usually not the bottleneck.

Scheduler Interactions¶

Pools interact with the Go scheduler in a few notable ways.

GOMAXPROCS¶

A pool with K=1000 workers exists, but only GOMAXPROCS of them run at once. The rest are parked (idle) or runnable (waiting for a CPU). If K >> GOMAXPROCS for CPU-bound work, the extra workers are wasted (and add scheduling overhead). For I/O-bound work, K >> GOMAXPROCS is fine — most workers are parked on I/O.

Work stealing¶

Go's scheduler does work stealing across P-queues. A pool worker that does work on P1 and then becomes idle will move to P1's local queue and be popped from there next time. Cache locality is OK in most cases.

Async preemption (Go 1.14+)¶

Goroutines can be preempted at "safe points" without explicit yields. This means a CPU-hogging pool worker won't starve other goroutines for long. Pre-1.14 Go could have CPU-bound workers monopolise a P; modern Go does not.

sysmon¶

The sysmon thread monitors for goroutines blocking in syscalls. When a pool worker enters a syscall (e.g., file read), sysmon may detach the worker's M (thread) so the P can run other goroutines. This is fine for pools doing I/O — the runtime handles thread expansion automatically.

Park/unpark of workers¶

When a pool worker has no task, it parks. When a task arrives, it's unparked. Each park/unpark is ~200ns. For very high task rates, this overhead accumulates. Some pools (ants with spin mode, if available) briefly spin before parking to avoid the cost — at the cost of CPU during the spin.

Tail Latency Analysis¶

Senior pool work often hinges on tail latency (p99, p999, p9999). What causes tail spikes in pool-using code?

Cause 1: Saturation¶

When the pool is at K and producers keep submitting, new submissions wait. The submitter blocks. Wait time grows. p99 spikes.

Detection: queue depth metric, submit-wait histogram.

Mitigation: raise K, drop on overload, scale out.

Cause 2: GC pause¶

Stop-the-world GC pauses all goroutines, including pool workers. A 100ms GC pause = 100ms added to whatever's in flight.

Detection: correlate p99 spikes with GC events (go_gc_duration_seconds).

Mitigation: reduce allocations, tune GOGC, use sync.Pool.

Cause 3: One slow task¶

If one task takes 10× longer than typical (a slow downstream, a slow query), that task blocks one worker for 10×. New tasks that need that worker's spot wait.

Detection: per-task duration histogram with very wide buckets.

Mitigation: per-task timeout, circuit breaker on downstream, separate pools for slow vs fast work.

Cause 4: Worker startup¶

If a worker just woke from idle and needs to warm up (cache fill, JIT, whatever), the first task on a freshly-started worker is slower. Idle expiry = repeated cold starts.

Detection: tail latency correlated with periods of low load.

Mitigation: disable idle expiry, longer expiry, or warm-state via tunny.

Cause 5: Scheduler latency¶

A goroutine that's runnable but not running waits for a P. If GOMAXPROCS is low or many CPU-bound goroutines compete, this wait grows.

Detection: scheduler trace (GODEBUG=schedtrace=1000) shows runnable count.

Mitigation: GOMAXPROCS tuning, fewer CPU-bound concurrent goroutines.

Cause 6: Lock contention¶

The pool's internal lock or your own task-internal lock causes serial execution. Effective concurrency drops; tail rises.

Detection: mutex profile.

Mitigation: sharded locks, lock-free structures, smaller critical sections.

Cause 7: Network jitter¶

For I/O-bound pools, p99 of the network call drives p99 of the task. Pool didn't cause it; downstream did.

Detection: pre-pool latency measurements.

Mitigation: retry with backoff, hedging, faster downstream.

Approach: bisect via metrics¶

When tail spikes appear:

1. Plot pool queue depth — saturation?
2. Plot per-task duration p99 — slow tasks?
3. Plot GC pauses — GC?
4. Plot scheduler latency — scheduler pressure?
5. Plot downstream latency — external?

The metric that correlates is the cause. Mitigation depends.

Failure Modes and Recovery¶

How does each library behave when things go wrong?

Pool worker panics¶

If panic recovery without crash is required: ants or pond, or wrap each task body manually.

Pool deadlock¶

A pool can deadlock if:

• Task A submits Task B and waits for B to complete; B requires the same pool worker A holds.
• All workers are blocked on a common resource that itself depends on a worker.

Detection: goroutine profile, watch for many goroutines blocked on the same channel/lock.

Mitigation: don't submit-and-wait inside a pool task using the same pool. Use a separate pool, or just inline the work.

Pool task leak¶

If a task hangs forever (no timeout), the worker is stuck. K-1 workers remain. Eventually all workers are stuck and the pool is dead.

Detection: pool running count not declining, queue growing.

Mitigation: per-task timeout, deadlock detection, restart the pool.

Pool teardown timeout¶

pool.ReleaseTimeout(30s) returns after 30 seconds even if tasks are still running. The remaining tasks are abandoned. If they hold resources (files, sockets), those leak.

Mitigation: ensure tasks respect ctx cancellation and bail out on shutdown.

Pool re-creation¶

If you Release a pool and then construct a new one, in-flight tasks of the old pool keep running on the old pool's workers until they finish. The new pool's workers are independent. There's no automatic migration.

Errgroup re-use¶

You cannot reuse an errgroup after Wait. If you do, the behaviour is undefined. Always make a new group per fan-out batch.

Race-Detector Behaviour¶

When running tests with -race, pools may emit race reports for shared state. Some notes:

Real races¶

If your task modifies a shared variable without sync (e.g., writes to a map), the race detector flags it. The pool is a red herring — the race is in your code.

Fix: protect shared state with mutex, atomic, or sync.Map.

Pool-internal races¶

Mature pool libraries (ants, tunny, workerpool) are race-free in their own code. If you see a race report inside the pool library, file a bug upstream.

Race-detector overhead¶

Running with -race is ~2-10× slower. Use it for tests, not production. Pool-using code is often slower under -race because of the high concurrency.

Senior Decision Process¶

A senior engineer doesn't just pick a tool — they document the process for picking. Here's a template.

Step 1: Quantify the workload¶

Document:

• Task rate (avg, peak, p99 burst).
• Per-task duration (median, p99).
• Per-task memory footprint.
• Failure modes (panic, timeout, partial).
• Dependencies (DB, HTTP, etc).

Step 2: Identify the bound¶

What is K rationing?

• CPU
• Downstream concurrency
• Memory
• File descriptors
• Or none — bounded by problem size

Step 3: List candidate tools¶

Apply the decision tree:

• Bounded by problem: raw.
• Single number, errors, ctx: errgroup.
• Unequal weight or cross-function: semaphore.
• High rate, fire-and-forget: ants.
• Worker state: tunny.
• Simple FIFO: workerpool.
• Metrics built-in: pond.

Step 4: Benchmark candidates¶

Write a microbenchmark with a synthetic workload that matches the real shape. Run all candidates. Measure:

• Throughput.
• p50/p99/p999 latency.
• CPU utilisation.
• Memory peak.
• GC overhead.

Step 5: Production load test¶

Deploy the candidate (behind a feature flag) and run a load test against a staging environment. Compare:

• Real downstream behaviour.
• Real network jitter.
• Real bursts.

Step 6: Pick, with rationale¶

Write up the decision in a design doc:

• Workload description.
• Candidates considered.
• Benchmark results.
• Production load test results.
• Final choice + K.
• What you'd change to invalidate the choice (e.g., "if rate exceeds 100k/sec, revisit").

Step 7: Monitor in production¶

Instrument with the metrics from earlier. Set alerts. Revisit if any metric drifts.

This process is overkill for small services. For services that handle real load, it's the minimum.

Real-World Architecture Case Studies¶

Three case studies of pool decisions in production-grade Go services.

Case Study 1: Distributed file uploader¶

Service. Uploads files from a queue to cloud object storage. ~200 uploads/sec average. File sizes: 1 KB to 100 MB.

Bounds. - S3 SDK has its own internal concurrency (~50 per client). - Network bandwidth: 1 Gbps egress. - Memory: 4 GiB pod; some buffering per upload.

Decision. errgroup with SetLimit(50), one upload per task. The S3 SDK already pools internally — adding a second pool would over-engineer.

Sizing rationale. S3's recommended concurrency is 50/client. Going higher provokes 503s. Going lower under-utilises bandwidth.

Production result. Stable at 200/sec average. Bursts to 500/sec saw errgroup back-pressure to the producer (Kafka). No drops, slightly higher queue lag during burst.

Lesson. errgroup is enough when the bound is a downstream limit. Don't add ants.

Case Study 2: Real-time bidding engine¶

Service. Receives bid requests from ad exchanges. ~30k bids/sec. Each bid: lookup user features in Redis (~2 ms), run a model (~3 ms), respond.

Bounds. - Bidder must respond in ≤80 ms (exchange timeout). - 32 cores. - Redis pool: 200 connections.

Decision. ants pool with K=4096, WithNonblocking=true, MaxBlockingTasks=2000.

Rationale. - 30k req/sec × 5 ms = 150 in-flight average; bursts to 1500+. K=4096 has headroom. - Spawn rate of 30k/sec from goroutines per request: measurable CPU on spawn. Pool amortises. - Non-blocking with drop: a bid that arrives during burst is dropped, not queued (queued bids time out before being processed; better to drop fast).

Production result. p99 latency 25ms, p999 60ms — under SLA. Drop rate <0.1% at peak.

Lesson. High-rate fire-and-forget is exactly where ants earns its place.

Case Study 3: Audio transcoding service¶

Service. Transcodes podcast episodes. ~10 episodes/sec. Each transcode: 30-180 seconds. CPU-bound.

Bounds. - 16 cores per pod. - ffmpeg uses 2-4 cores per transcode (depending on quality).

Decision. Tunny with K=4, each worker holding a pre-warmed ffmpeg context.

Rationale. - 4 transcodes × 4 cores = 16 cores. Full utilisation. - ffmpeg's setup cost (~5s per spawn) is amortised across many transcodes per worker. - Tunny's Process(payload) any returns the transcode result naturally.

Production result. Throughput 4 transcodes/sec sustained; with 1 active pod and 4 workers, queue grows during peaks. Scaled to 3 pods, queue stays clear.

Lesson. Tunny is the right answer when warm state and worker model align.

Anti-Patterns at Senior Level¶

Beyond the obvious anti-patterns from junior/middle levels.

Anti-pattern: Pool with no observability¶

A pool in production without metrics is a black box. You cannot tell if it's saturated, slow, or starving. Always instrument.

Anti-pattern: K hardcoded for one environment¶

pool, _ := ants.NewPool(64) works on the dev laptop. In production, 64 may be wrong for any of: 8-core pod (too high), 64-core pod (too low), Kubernetes-throttled pod (way too high).

Make K configurable. Tune per environment.

Anti-pattern: Pool used as task-class router¶

pool.Submit(func() {
    if isFast(t) { fastPath(t) } else { slowPath(t) }
})

Slow tasks block workers; fast tasks queue behind them. Effective concurrency for fast tasks drops.

Better: two pools, one for fast, one for slow. Routed by task class.

Anti-pattern: Shared pool for unrelated subsystems¶

var globalPool = ants.NewPool(1000)
// used by auth, billing, search, ...

One subsystem's noisy load starves others. Each subsystem should have its own pool, sized for its bounds.

Anti-pattern: Pool inside a request handler¶

func handler(w, r) {
    pool, _ := ants.NewPool(10)
    defer pool.Release()
    // ...
}

New pool per request = pool setup/teardown cost per request. Pools should be long-lived. Construct at service init, reuse.

Anti-pattern: Pool sized for peak, not for normal¶

A pool sized for peak (e.g., K=10000 for occasional bursts) holds 10000 workers idle most of the time. Each worker is 3 KB; 30 MB at idle.

Better: smaller pool with idle expiry, allows pool to scale up under load. Or auto-tune.

Anti-pattern: Submitting and waiting in the same goroutine¶

pool.Submit(...) // task A
pool.Submit(...) // task B
pool.Submit(...) // task C
// no wait!
// returns

If the function returns before tasks finish, who waits? Without a WaitGroup or join, the tasks run after the caller goes away — fine for fire-and-forget but a bug if you needed the results.

Anti-pattern: Errgroup leak¶

g, _ := errgroup.WithContext(ctx)
g.Go(longRunning)
return // never called Wait!

The errgroup spawns goroutines that may run for a long time. Without Wait, the parent function returns, but the goroutines linger. If they capture variables that should be released, they hold them.

Always Wait. Always.

Designing for the Long Run¶

Pools live in services that live for years. Decisions you make today haunt you tomorrow.

Versioned guidelines¶

Pick a tool and stick with it for your team. Switching libraries every year is churn. Pick errgroup as default; pick one third-party pool (e.g., ants) as the "we use this when we need a pool" choice. Document it.

Migration paths¶

When a pool no longer fits, plan the migration. Don't let a misfit pool live in production for years.

Dependency hygiene¶

Audit your pool library annually. Read its CHANGELOG. Note breaking changes. Plan upgrades.

Observability hygiene¶

Every pool should expose at least the five core metrics (running, queued, submitted, panic, drop). Standardise the names across services.

Onboarding new engineers¶

A new hire sees ants.NewPool(...) and wonders why. Have a one-page document: "Why we use ants in service X." This is more valuable than any comment in the code.

Capacity planning¶

Pool sizing should be part of capacity planning. Document the K, the rationale, and the scenarios where you'd raise/lower it.

Self-Assessment¶

Senior-level pool literacy looks like this:

• I can read a pprof CPU profile and identify pool overhead.
• I can compute K from Little's Law given throughput and latency targets.
• I can name the trade-offs between blocking and non-blocking submit.
• I can explain LIFO vs FIFO worker selection and when each is right.
• I can identify a panic-handler-requiring workload.
• I can write a benchmark that distinguishes ants from errgroup.
• I have rejected a pool in code review with measurements.
• I have approved a pool in code review with measurements.
• I have migrated a service between pool libraries with no production incidents.
• I know what GODEBUG=schedtrace=1000 does and have used it.
• I know the difference between runtime.NumCPU() and runtime.GOMAXPROCS(0) in containers.
• I have at least one pool in production with full metrics.
• I have removed at least one pool that wasn't earning its keep.
• I can argue both sides of the "should we use ants here" debate.

If yes — professional.md covers the operational side.

Summary¶

• Pool internals matter at high contention: locking strategy, allocation per task, scheduling fairness, memory footprint.
• ants has the lowest Submit overhead via worker stack (LIFO) and optionally lock-free queues.
• tunny has the best worker-state model but loses on type safety and contention.
• workerpool is the smallest API but lags on high-contention throughput.
• pond combines features (groups, metrics) and sharded queues.
• Allocations per task is dominated by closure captures; Invoke-style avoids them.
• Scheduling fairness varies: LIFO (ants) for cache locality; FIFO (workerpool, tunny) for predictability.
• Memory footprint is small (few MB even for large pools) but visible in tight environments.
• Tail latency drivers: saturation, GC pause, slow tasks, worker startup, scheduler.
• Pool failure modes: panic, deadlock, task leak, teardown timeout. Choose libraries with mitigations.
• The senior decision process is: quantify → identify bound → list candidates → benchmark → load test → pick → monitor.

Appendix S1: Internals of ants — A Walk-Through¶

A senior engineer should be able to read the source of their dependencies. Let's walk through ants's Submit path.

ants's Pool struct (simplified)¶

type Pool struct {
    capacity      int32
    running       int32
    lock          sync.Locker
    workers       workerQueue
    state         int32
    cond          *sync.Cond
    workerCache   sync.Pool
    waiting       int32
    purgeDone     int32
    stopPurge     context.CancelFunc
    ticktockDone  int32
    stopTicktock  context.CancelFunc
    options       *Options
}

Key fields: - capacity: the K. - running: count of active workers. - workers: a queue of idle workers (stack or loop queue). - cond: condition variable for blocking submitters. - workerCache: a sync.Pool for recycling worker structs.

Submit path¶

func (p *Pool) Submit(task func()) error {
    if p.IsClosed() {
        return ErrPoolClosed
    }
    if w := p.retrieveWorker(); w != nil {
        w.inputFunc(task)
        return nil
    }
    return ErrPoolOverload
}

The work is in retrieveWorker. Let's read it:

func (p *Pool) retrieveWorker() (w worker) {
    spawnWorker := func() {
        w = p.workerCache.Get().(*goWorker)
        w.run()
    }

    p.lock.Lock()

    w = p.workers.detach()
    if w != nil {
        p.lock.Unlock()
        return
    }

    if capacity := p.Cap(); capacity == -1 || capacity > p.Running() {
        p.lock.Unlock()
        spawnWorker()
        return
    }

    // Non-blocking case
    if p.options.Nonblocking {
        p.lock.Unlock()
        return
    }

retry:
    // Blocking with max blocking tasks
    if p.options.MaxBlockingTasks != 0 && p.Waiting() >= p.options.MaxBlockingTasks {
        p.lock.Unlock()
        return
    }
    p.addWaiting(1)
    p.cond.Wait()
    p.addWaiting(-1)
    if p.IsClosed() {
        p.lock.Unlock()
        return
    }

    if w = p.workers.detach(); w == nil {
        if p.Running() < p.Cap() {
            p.lock.Unlock()
            spawnWorker()
            return
        }
        goto retry
    }
    p.lock.Unlock()
    return
}

Walk through:

1. Lock the mutex.
2. Try to pop an idle worker from the queue.
3. If got one, unlock and return it.
4. Else, if pool not at capacity, spawn a new worker.
5. Else, if non-blocking, return nil (the caller gets ErrPoolOverload).
6. Else, block on the condition variable until a worker frees up.
7. If MaxBlockingTasks is set and we're at that limit, return nil immediately.
8. Loop.

This is straightforward; the design is solid. The single mutex is the contention point at high rate. The "loop queue" variant replaces the worker queue with a CAS-based ring buffer to reduce contention.

Worker run loop¶

func (w *goWorker) run() {
    w.pool.addRunning(1)
    go func() {
        defer func() {
            w.pool.addRunning(-1)
            w.pool.workerCache.Put(w)
            if p := recover(); p != nil {
                if ph := w.pool.options.PanicHandler; ph != nil {
                    ph(p)
                }
            }
            w.pool.cond.Signal()
        }()

        for f := range w.task {
            if f == nil {
                return
            }
            f()
            if ok := w.pool.revertWorker(w); !ok {
                return
            }
        }
    }()
}

Walk through:

1. Increment running counter.
2. Spawn the worker goroutine.
3. On exit: decrement counter, return self to cache, recover panic (call handler), signal cond for waiters.
4. Loop: receive a task from the channel, execute it, return self to idle queue.
5. If revertWorker returns false (pool closed or worker expired), exit.

The worker is a long-lived goroutine that loops on w.task (a channel). New tasks are pushed via w.inputFunc(task), which sends on w.task. The worker, between tasks, sits parked on the receive.

The IdleTimeout¶

When a worker is returned to the idle queue, its recycleTime is updated. A separate purge goroutine periodically scans the queue:

func (p *Pool) purgeStaleWorkers(ctx context.Context) {
    heartbeat := time.NewTicker(p.options.ExpiryDuration)
    defer heartbeat.Stop()
    for {
        select {
        case <-ctx.Done():
            return
        case <-heartbeat.C:
        }
        p.lock.Lock()
        staleWorkers := p.workers.refresh(p.options.ExpiryDuration)
        p.lock.Unlock()
        for i := range staleWorkers {
            staleWorkers[i].finish()
        }
    }
}

Workers idle longer than ExpiryDuration (default 1s) are finished (their channel closed, they exit the run loop).

Why this matters for choice¶

• The single mutex is fine for moderate load.
• The condition variable for blocking submitters is contended at high producer count.
• Idle expiry can cause oscillation: workers expire, then must be re-spawned on the next burst.
• MaxBlockingTasks is essential for bounded queues — without it, the waiting list grows unbounded.

Knowing this lets you choose options deliberately, not by copy-paste.

Appendix S2: Internals of errgroup¶

Compare with errgroup's much smaller surface area.

type Group struct {
    cancel  func(error)
    wg      sync.WaitGroup
    sem     chan token
    errOnce sync.Once
    err     error
}

type token struct{}

func (g *Group) SetLimit(n int) {
    if n < 0 {
        g.sem = nil
        return
    }
    if len(g.sem) != 0 {
        panic(fmt.Errorf("errgroup: modify limit while %v goroutines in the group are still active", len(g.sem)))
    }
    g.sem = make(chan token, n)
}

func (g *Group) Go(f func() error) {
    if g.sem != nil {
        g.sem <- token{}
    }
    g.wg.Add(1)
    go func() {
        defer g.done()
        if err := f(); err != nil {
            g.errOnce.Do(func() {
                g.err = err
                if g.cancel != nil {
                    g.cancel(g.err)
                }
            })
        }
    }()
}

func (g *Group) done() {
    if g.sem != nil {
        <-g.sem
    }
    g.wg.Done()
}

func (g *Group) Wait() error {
    g.wg.Wait()
    if g.cancel != nil {
        g.cancel(g.err)
    }
    return g.err
}

Walk through:

• SetLimit(n) creates a channel of capacity n. This is the semaphore.
• Go(f) first sends on the channel (blocks if full), then spawns a goroutine.
• The goroutine, when done, receives from the channel (freeing a slot).
• errOnce ensures only the first error is captured.
• cancel cancels the shared ctx.
• Wait() waits for all goroutines, returns the first error.

This is ~50 lines. It's minimal and elegant. Compare with ants's ~500 lines of pool implementation.

Why errgroup spawns per call¶

Each Go spawns a goroutine. There is no worker reuse. The semaphore just bounds how many run at once. The goroutine pays the spawn cost; the semaphore pays the bound.

This is fine when spawn cost is negligible (rare to use 100k+ goroutines/sec on the same group). When it's not, ants's worker reuse wins.

Appendix S3: Internals of tunny¶

Tunny's worker model is unique.

type Pool struct {
    queuedJobs   int64
    ctor         func() Worker
    workers      []*workerWrapper
    reqChan      chan workRequest
    workerMut    sync.Mutex
}

type Worker interface {
    Process(any) any
    BlockUntilReady()
    Interrupt()
    Terminate()
}

The pool has: - A ctor function that creates each worker. - A slice of workerWrapper (each owns a goroutine and a Worker instance). - A reqChan where Process sends requests.

type workRequest struct {
    jobChan chan<- any
    retChan <-chan any
    interrupt chan<- struct{}
}

Each work request is a triple: the worker's input channel, the worker's output channel, and an interrupt channel.

Process¶

func (p *Pool) Process(payload any) any {
    atomic.AddInt64(&p.queuedJobs, 1)
    request, open := <-p.reqChan
    if !open {
        panic(ErrPoolNotRunning)
    }
    request.jobChan <- payload
    payload, open = <-request.retChan
    if !open {
        panic(ErrWorkerClosed)
    }
    atomic.AddInt64(&p.queuedJobs, -1)
    return payload
}

1. Receive a worker's "I'm idle" request from reqChan.
2. Send the payload to that worker's job channel.
3. Wait for the worker to return the result.
4. Return.

The flow is: workers broadcast their idleness; clients pick up an idle worker via reqChan; client and worker pair up.

Worker loop¶

func (w *workerWrapper) run() {
    w.worker.BlockUntilReady()
    w.reqChan <- workRequest{
        jobChan:   w.jobChan,
        retChan:   w.retChan,
        interrupt: w.interruptChan,
    }
    for {
        select {
        case payload := <-w.jobChan:
            result := w.worker.Process(payload)
            select {
            case w.retChan <- result:
            case <-w.interruptChan:
            }
            w.worker.BlockUntilReady()
            w.reqChan <- workRequest{...}
        case <-w.closeChan:
            w.worker.Terminate()
            return
        }
    }
}

The worker: 1. Calls BlockUntilReady on the worker (custom init). 2. Sends a workRequest to reqChan, advertising itself as idle. 3. Waits for a payload on jobChan. 4. Processes it. 5. Returns the result on retChan. 6. Loops.

Worker.Process(payload) is your code. It has access to the worker struct (so per-worker state lives there).

Why this is unique¶

• Each worker has its own state via the constructor.
• The dispatch is via reqChan, not a shared queue. The dispatch is "first idle worker wins" — slightly different fairness than a queue.
• There is no queue of pending tasks. If all workers are busy, Process blocks on <-p.reqChan.

For workloads where worker state matters, tunny is the only library that models it directly. For workloads where state doesn't matter, the queue model (ants, workerpool) is fine.

Cost¶

• One Process call: ~600 ns at low contention.
• Under high contention (many producers): contention on reqChan. The reqChan is single-channel; all producers contend on the receive operation.

For high-rate workloads with worker state, this can be a bottleneck. Mitigation: split into multiple smaller pools.

Appendix S4: Internals of workerpool¶

The smallest of the libraries. Source is ~200 lines.

type WorkerPool struct {
    maxWorkers   int
    taskQueue    chan func()
    workerQueue  chan func()
    stoppedChan  chan struct{}
    stopSignal   chan struct{}
    waitingQueue deque.Deque[func()]
    stopLock     sync.Mutex
    stopOnce     sync.Once
    stopped      bool
    waiting      int32
    wait         bool
}

func (p *WorkerPool) Submit(task func()) {
    if task != nil {
        select {
        case p.taskQueue <- task:
        default:
            // Slow path: enqueue in deque
        }
    }
}

func (p *WorkerPool) dispatch() {
    defer close(p.stoppedChan)
    timeout := time.NewTimer(idleTimeout)
Loop:
    for {
        // If there are tasks in the waiting queue, push to workers
        if p.waitingQueue.Len() != 0 {
            ...
        }
        select {
        case task, ok := <-p.taskQueue:
            if !ok { break Loop }
            select {
            case p.workerQueue <- task:
            default:
                if p.numWorkers < p.maxWorkers {
                    go startWorker(task, p.workerQueue)
                    p.numWorkers++
                } else {
                    p.waitingQueue.PushBack(task)
                    atomic.StoreInt32(&p.waiting, int32(p.waitingQueue.Len()))
                }
            }
        ...
        }
    }
}

There's a single dispatcher goroutine that: - Reads tasks from taskQueue (where Submit sends). - Sends them to workerQueue (where workers receive). - If no worker is ready and we're below max, spawn a new one. - Otherwise queue in waitingQueue (a Deque).

This single-dispatcher design has a clear bottleneck: the dispatcher. At very high rates, it cannot keep up. For most workloads, it's fine.

Worker¶

func startWorker(task func(), workerQueue chan func()) {
    task()
    go worker(workerQueue)
}

func worker(workerQueue chan func()) {
    for task := range workerQueue {
        task()
    }
}

The worker is a simple for task := range channel loop. Workers exit when the channel is closed.

Verdict¶

workerpool is a fine choice for moderate-throughput, simple workloads. Read its source in 15 minutes. Audit it. Use it.

Appendix S5: Custom Pool — Build Your Own¶

Once you understand the libraries, you can build your own pool in 50 lines. Here's one for educational purposes.

type Pool struct {
    tasks   chan func()
    wg      sync.WaitGroup
    stopped atomic.Bool
}

func New(workers, queue int) *Pool {
    p := &Pool{tasks: make(chan func(), queue)}
    for i := 0; i < workers; i++ {
        p.wg.Add(1)
        go p.work()
    }
    return p
}

func (p *Pool) work() {
    defer p.wg.Done()
    for t := range p.tasks {
        func() {
            defer func() {
                if r := recover(); r != nil {
                    log.Printf("pool worker panic: %v", r)
                }
            }()
            t()
        }()
    }
}

func (p *Pool) Submit(t func()) error {
    if p.stopped.Load() {
        return errors.New("pool stopped")
    }
    select {
    case p.tasks <- t:
        return nil
    default:
        return errors.New("pool full")
    }
}

func (p *Pool) SubmitBlocking(ctx context.Context, t func()) error {
    if p.stopped.Load() {
        return errors.New("pool stopped")
    }
    select {
    case p.tasks <- t:
        return nil
    case <-ctx.Done():
        return ctx.Err()
    }
}

func (p *Pool) Stop() {
    if !p.stopped.CompareAndSwap(false, true) {
        return
    }
    close(p.tasks)
    p.wg.Wait()
}

50 lines. Has: - Fixed worker count. - Bounded queue. - Non-blocking and blocking submit. - Panic recovery. - Graceful stop with drain.

Missing: - Idle worker expiry. - Dynamic resize. - Metrics. - Worker state.

For 80% of use cases where you want a "custom" pool, this is enough. If you find yourself wanting these features, you'd reinvent ants — at which point, just use ants.

When to use your own pool¶

• You need a specific feature (priority, multi-queue, custom backpressure).
• You want to fully audit the dependency surface.
• Educational.

In production: prefer the library if it covers your needs. Custom code is technical debt unless it's specifically uncommon.

Appendix S6: Concurrency-Safe Result Collection¶

A senior topic: how to collect results from N concurrent tasks safely and efficiently.

Option 1: Indexed slice (most common)¶

results := make([]Result, len(items))
g, _ := errgroup.WithContext(ctx)
g.SetLimit(K)
for i, item := range items {
    i, item := i, item
    g.Go(func() error {
        r, err := work(item)
        if err != nil { return err }
        results[i] = r
        return nil
    })
}
return results, g.Wait()

Each goroutine writes to its own slice index. No locking required because indexes don't overlap. The runtime guarantees that the write is atomic if the value is word-sized; for struct values, the entire struct is copied, which is a single memory copy with no concurrent partial writes.

The race detector confirms this is safe. It's the canonical pattern.

Option 2: Channel of results¶

out := make(chan Result, len(items))
g, _ := errgroup.WithContext(ctx)
g.SetLimit(K)
for _, item := range items {
    item := item
    g.Go(func() error {
        r, err := work(item)
        if err != nil { return err }
        out <- r
        return nil
    })
}
go func() { g.Wait(); close(out) }()

var results []Result
for r := range out {
    results = append(results, r)
}

Tasks send results to a channel. A consumer collects them. Order is non-deterministic.

Use when: - You want to stream results out as they complete. - Order doesn't matter. - You want to consumer-side terminate early (skip remaining results).

Option 3: sync.Mutex + shared slice¶

var mu sync.Mutex
var results []Result
g, _ := errgroup.WithContext(ctx)
g.SetLimit(K)
for _, item := range items {
    item := item
    g.Go(func() error {
        r, err := work(item)
        if err != nil { return err }
        mu.Lock()
        results = append(results, r)
        mu.Unlock()
        return nil
    })
}
return results, g.Wait()

Less efficient than Option 1 because of lock contention. Use only when input order doesn't correspond to output order (e.g., filtering: some inputs produce no output).

Option 4: sync.Map¶

var results sync.Map
g, _ := errgroup.WithContext(ctx)
g.SetLimit(K)
for _, item := range items {
    item := item
    g.Go(func() error {
        r, err := work(item)
        if err != nil { return err }
        results.Store(item.Key, r)
        return nil
    })
}
g.Wait()

// later: iterate or look up
results.Range(func(k, v any) bool { ... return true })

sync.Map is optimised for read-mostly access. For pure write-many, it's slower than Option 3. Use when you'll do many concurrent reads of the results map after.

Option 5: Atomic counter + result slice¶

For counting:

var count atomic.Int64
g, _ := errgroup.WithContext(ctx)
g.SetLimit(K)
for _, item := range items {
    item := item
    g.Go(func() error {
        if didWork(item) {
            count.Add(1)
        }
        return nil
    })
}
g.Wait()
fmt.Println("did work:", count.Load())

Use atomic for simple aggregates. For complex aggregates (sums, histograms), need lock or per-goroutine accumulators + final reduce.

Recommendation¶

Use Option 1 (indexed slice) whenever the output order matches input order. Use Option 2 (channel) for streaming. Use Option 3 (mutex slice) sparingly. Avoid Option 4 unless reads dominate.

Appendix S7: The "When Goroutines Are Too Cheap" Paradox¶

A subtle senior observation: goroutines are too cheap. Their cheapness encourages a class of bug.

The bug¶

Because spawning a goroutine costs almost nothing, developers reach for go f() reflexively. They write code that spawns goroutines they never wait for, goroutines that pile up, goroutines whose lifecycle is unclear. Each one alone is cheap; collectively they accumulate.

The bug is not "goroutines are expensive" — it's "concurrency is cognitively expensive." Each spawn is a new path through the program; the more paths, the harder to reason.

Why pools help¶

A pool makes goroutines an explicit, scarce resource. You cannot accidentally spawn 100,000 — the pool has K slots and rejects beyond that. This restraint is the pool's main cognitive benefit, beyond the performance angle.

Compare:

// Casual code
go logEvent(e)
go updateMetrics(m)
go callExternal(api)
// ...

Versus:

// Pool-disciplined code
backgroundPool.Submit(func() { logEvent(e) })
backgroundPool.Submit(func() { updateMetrics(m) })
externalPool.Submit(func() { callExternal(api) })
// ...

The second form has explicit scarce resources. You'll think twice before adding another Submit line.

When this matters¶

In large codebases with many engineers, the discipline of "spawn through a pool" prevents accidental goroutine explosions. In small codebases with a few engineers who all know the code, it's overkill.

The hidden cost¶

The cost is exactly the dependency and the boilerplate. Worthwhile if your team needs the discipline; over-engineering if it doesn't.

Appendix S8: Pool-Specific Profiling Tricks¶

Trick 1: Submit-rate histogram¶

Add a custom Prometheus histogram counting Submit calls per second per pool. Useful for detecting unexpected spikes.

Trick 2: Per-task duration tagging¶

Wrap your task closure with a duration tag:

pool.Submit(func() {
    start := time.Now()
    work(item)
    taskDurationHistogram.WithLabelValues("type", item.Type).Observe(time.Since(start).Seconds())
})

Split tail latency by task class. The slow class is your tail.

Trick 3: Queue depth alert¶

Alert when pool.Waiting() > threshold for X minutes. Saturation early-warning.

Trick 4: Trace-driven analysis¶

Use runtime/trace:

trace.Start(f)
defer trace.Stop()
// ... do work ...

Open in go tool trace. You can see exactly which goroutine ran on which P, when it blocked, when it was unparked. Invaluable for understanding pool scheduling behaviour.

Trick 5: Goroutine-by-state breakdown¶

pprof goroutine ?debug=2 shows every goroutine's current state. For a pool of 1000 workers, you should see 1000 in "select (chan receive)" state. If many are in "chan send" (trying to submit results) or "chan receive" (waiting for input), there's a bottleneck downstream.

Appendix S9: When to Vendor a Pool Library¶

Vendoring (copying the library into your repo's vendor/ or internal/) is a tradeoff:

• Pro: Full control, version pinning, no upstream surprise.
• Con: You own the maintenance.

When to vendor:

• Critical service whose pool behaviour is load-bearing.
• Library has been abandoned but works.
• You need to patch a bug upstream won't accept.

When not to:

• Active, well-maintained library.
• Stable releases.
• No bugs that bite you.

For ants and pond, don't vendor. For tunny (less active), vendor if you depend deeply.

Appendix S10: Pool Library Reading¶

When you adopt a third-party pool, read the source. Not all of it; the Submit path and the worker run loop.

Source size:

• ants: ~3000 lines (multiple files). Read pool.go (Submit) and worker.go (run loop).
• tunny: ~500 lines, one file. Read it all.
• workerpool: ~400 lines. Read it all.
• pond: ~1000 lines. Read the core.

The investment is one afternoon. Pays off when you debug a production issue.

Appendix S11: Pool Bug Stories¶

A few cautionary tales.

Story 1: The pool that didn't drain¶

A team configured ants.NewPool(100) but never called Release(). The pool kept its workers alive forever. Each test run leaked 100 goroutines. After 1000 test runs, the test process had 100,000 goroutines. CI started failing with OOM.

Fix: defer pool.Release(). Tests stable.

Story 2: The submit error nobody checked¶

pool.Submit(func() { ... }) // returns error; ignored

Under load, Submit returned ErrPoolOverload. The task was silently dropped. The team noticed a percentage of tasks "never happened" only after a customer complained.

Fix: check the error. Either retry, fall back, or alarm.

Story 3: The pool of zero¶

K := os.Getenv("POOL_K") // returns "" if unset
n, _ := strconv.Atoi(K)   // returns 0 if "" or unparseable
pool, _ := ants.NewPool(n)

When env var is unset, K=0. ants.NewPool(0) returns an error (or panics, depending on version). The team noticed at deploy time when the service refused to start.

Fix: validate K. Default to a sane value if env var is missing.

Story 4: Pool reuse across tests¶

Tests shared a global pool. Test A submits 100 tasks; Test B starts before A's tasks finish; A's tasks pollute B's assertions.

Fix: per-test pool, or test pool drained between tests.

Story 5: The panic that escaped¶

A pool with no panic handler. One task hit a divide-by-zero. The worker crashed. The pool's spawned goroutine died, no replacement spawned. Over hours, the pool's worker count drifted down. Eventually all workers gone; pool dead.

Fix: panic handler that logs and lets the pool re-spawn the worker.

Story 6: The pool that scaled the wrong way¶

A team auto-scaled K based on CPU. CPU went high → K decreased (to reduce CPU). CPU went low → K increased (to do more work). The control loop was backwards: high CPU triggers a smaller pool, which means less work, which means less CPU. Pool drove itself to zero.

Fix: read the control loop carefully; auto-scaling is subtle.

Story 7: Deadlock by self-submit¶

A pool task submits to the same pool and waits for the result. If the pool is full of such tasks, all are waiting for workers that don't exist. Deadlock.

Fix: never submit-and-wait to the same pool. Use a separate pool, or inline the work.

Appendix S12: Senior Reading List¶

Beyond the libraries' READMEs:

• "Scalable Go Scheduler Design" — original design doc.
• Dmitry Vyukov's papers on lock-free programming.
• "Locking in WebKit" — general intuition about lock contention.
• "Mechanical Sympathy" blog (Martin Thompson) — cache and false sharing.
• The Go memory model document.
• "The Many Faces of Mutex" — talks on mutex internals.

Appendix S13: Three Years of Pool Decisions¶

A meta-observation: looking at three years of pool-related PRs across various services, the pattern of decisions follows a pattern:

• Year 1: team adopts ants/tunny enthusiastically.
• Year 2: team realises many places don't need a pool, migrates to errgroup.
• Year 3: team settles into a stable pattern: errgroup for most, ants for specific hot paths, tunny for worker-state.

The lesson: the maturity curve is universal. New teams over-adopt; mature teams under-adopt then settle. If you're new, anticipate this. Don't fight the curve; just accelerate to the middle.

Appendix S14: Pools and Distributed Systems¶

In distributed systems, the "pool" question is bigger than one process.

Per-process pool vs per-cluster bound¶

If your cluster has 10 replicas, each with K=50, the cluster has 500 concurrent. The downstream sees 500. If downstream allows 100, you exceed by 5x.

Mitigations:

• Static per-process K = limit / replicas. Coordinate at deploy time.
• Distributed rate limiter (Redis, dedicated service).
• Service-mesh-level rate limiting (Envoy, Linkerd).

Pool as bulkhead¶

In microservices architecture, each downstream has its own pool. The pool isolates failures: if downstream A is slow, only the pool for A saturates; pool B is unaffected. This is the bulkhead pattern.

type DownstreamPools struct {
    userServicePool   *ants.Pool
    billingPool       *ants.Pool
    notifsPool        *ants.Pool
}

Each pool sized for its downstream. One slow service doesn't block calls to others.

Pool sizing under partial failure¶

When a downstream is degraded, in-flight calls take longer. By Little's Law, more concurrency is needed to maintain throughput. Pool sized for "happy path" gets saturated under degradation. Some teams oversize K during operations; others retreat to circuit-breaker patterns.

Appendix S15: The Senior's Checklist for Pool Code Review¶

When reviewing pool-using code, in addition to the junior/middle checklist:

• Is the pool long-lived (not per-request)?
• Does it have a panic handler?
• Does Submit's return error get checked?
• Is K configurable (env var or config)?
• Is there a metric for in-flight, queue, drops?
• Is there an alert for saturation?
• What happens on SIGTERM (drain)?
• Does the pool live in a separate file with its own tests?
• Is there a benchmark showing the pool's benefit over errgroup?
• What's the rollback plan if the pool causes issues?

Most pool code in real codebases fails 3+ of these. The code review pulls them out, one by one, with patient explanation.

Appendix S16: Looking Ahead¶

After senior, the next level is operational. professional.md covers SLAs, observability for pools, third-party risk, and the build vs adopt question. The remaining files are reference: specification of APIs, interview Q&A, hands-on tasks, bugs, optimisations.

A senior engineer's relationship with pools is: trust but verify, default to errgroup, justify everything, measure everything, document everything. The next level is to operationalise that into team practice.

Appendix S17: Comparative Workloads¶

We will now examine eight workloads in full depth, comparing all four tools (raw, errgroup, semaphore, pool) for each. This is the data behind the decision tree.

Workload S17.1: Web crawler¶

Crawl 1M URLs. Each fetch: 200 ms median, p99 800 ms. Politeness: 10 concurrent per host. There are 10k hosts.

Constraints: - HTTP client connection pool. - Per-host rate limit (politeness). - 16 GiB memory budget.

Analysis: - 10 concurrent per host × 10k hosts = up to 100k concurrent in theory. Memory doesn't allow it. - Bound by memory: each in-flight has ~1 KB response buffer + connection state ≈ 5 KB. Budget 16 GiB / 5 KB ≈ 3M. Memory isn't the binding constraint. - Bound by file descriptors: ulimit -n typically 65536. So K ≤ 32k. - Real-world choice: K = 1000 globally, with per-host semaphore of 10.

Tool: errgroup for fan-out + sync.Map of semaphore.Weighted(10) per host.

hostSems := sync.Map{}

getSem := func(host string) *semaphore.Weighted {
    if v, ok := hostSems.Load(host); ok { return v.(*semaphore.Weighted) }
    s := semaphore.NewWeighted(10)
    v, _ := hostSems.LoadOrStore(host, s)
    return v.(*semaphore.Weighted)
}

g, ctx := errgroup.WithContext(ctx)
g.SetLimit(1000)
for _, u := range urls {
    u := u
    g.Go(func() error {
        host := extractHost(u)
        sem := getSem(host)
        if err := sem.Acquire(ctx, 1); err != nil { return err }
        defer sem.Release(1)
        return fetch(ctx, u)
    })
}
return g.Wait()

Two bounds layered: global (errgroup, 1000) and per-host (semaphore, 10).

Workload S17.2: Real-time bidder (revisited in depth)¶

Recall: 30k bids/sec, 80ms p99 SLA, GPU-bound inference.

Question: ants vs errgroup vs custom?

• errgroup with SetLimit(K): each bid spawns a goroutine. At 30k/sec, spawn rate is high. Measure: ~5-8% CPU on goroutine creation.
• ants with K=4096: workers reused. Submit ~250ns. ~1-2% CPU on dispatch.
• Custom pool: same as ants but with priority queue (high-value bids first).

Choice: ants for general; custom if priority matters.

Sizing K: - Inference latency 5ms, throughput 30k/sec → Little's Law: K = 30000 × 0.005 = 150 active. - For burst (3× peak): K = 450 active. - Pool size 4096 has 10× headroom — useful for tail tolerance.

K=4096 ants. Non-blocking with drop on overflow. Drop rate alert >1%.

Workload S17.3: Database migration¶

A one-off migration: read 100M rows from old DB, transform, write to new DB. Run during maintenance window (2 hours).

Constraints: - Source DB: 100 concurrent reads. - Destination DB: 50 concurrent writes. - 32 cores, 64 GiB memory.

Analysis: - 100M rows in 7200s = 14k rows/sec target. - Read latency ~5ms, write latency ~20ms → 14k × 0.020 = 280 in-flight writes; bound at 50. Write is bottleneck. - Need to read at 14k/sec; 14k × 0.005 = 70 in-flight reads; bound at 100. Fine.

Tool: pipeline of two errgroups (read stage, write stage), connected by a bounded channel.

const readK, writeK = 100, 50
ch := make(chan Row, 1000)

g, ctx := errgroup.WithContext(ctx)
g.Go(func() error {
    defer close(ch)
    subg, ctx := errgroup.WithContext(ctx)
    subg.SetLimit(readK)
    for id := range rowIDs {
        id := id
        subg.Go(func() error {
            row, err := source.Read(ctx, id)
            if err != nil { return err }
            select { case ch <- row: case <-ctx.Done(): return ctx.Err() }
            return nil
        })
    }
    return subg.Wait()
})

g.Go(func() error {
    subg, ctx := errgroup.WithContext(ctx)
    subg.SetLimit(writeK)
    for row := range ch {
        row := row
        subg.Go(func() error { return dest.Write(ctx, row) })
    }
    return subg.Wait()
})

return g.Wait()

No third-party pool needed. The whole thing in errgroup.

Workload S17.4: Continuous health checker¶

A service health-checks 1000 endpoints every 30 seconds. Each check: ~100 ms HTTP call.

Constraints: - Memory budget: 1 GiB (modest service). - 1000 endpoints × 1 check/30s = 33 checks/sec average; can burst at start of each interval.

Tool: errgroup per interval, with SetLimit(50). At start of each 30s interval, fan out:

ticker := time.NewTicker(30 * time.Second)
for range ticker.C {
    g, ctx := errgroup.WithContext(ctx)
    g.SetLimit(50)
    for _, ep := range endpoints {
        ep := ep
        g.Go(func() error { return check(ctx, ep) })
    }
    g.Wait()
}

1000 / 50 = 20 batches of 50; at 100ms each, the whole pass takes ~2 seconds. Plenty of slack before the next 30s tick.

ants here would be overkill. Spawn rate ~33/sec is trivial.

Workload S17.5: Image processing pipeline¶

A user uploads a photo; service produces 5 thumbnail sizes. Volume: 100 uploads/sec.

Constraints: - Per-image: 100ms CPU-bound resize. - 32 cores. - Memory: 2 GiB per pod.

Analysis: - 100 uploads × 5 sizes = 500 resize tasks/sec. - CPU bound: K = NumCPU = 32. - 500/sec × 0.1s = 50 active; bound at 32; queue depth ~20 average.

Tool: errgroup per upload (5 tasks), with a per-process semaphore of 32.

var resizeSem = semaphore.NewWeighted(32)

func process(ctx context.Context, img Image) ([]Image, error) {
    results := make([]Image, 5)
    g, ctx := errgroup.WithContext(ctx)
    for i, size := range sizes {
        i, size := i, size
        g.Go(func() error {
            if err := resizeSem.Acquire(ctx, 1); err != nil { return err }
            defer resizeSem.Release(1)
            r, err := resize(img, size)
            if err != nil { return err }
            results[i] = r
            return nil
        })
    }
    return results, g.Wait()
}

The cross-handler semaphore enforces the CPU bound globally. The per-handler errgroup propagates errors and ctx.

Workload S17.6: Periodic batch summarizer¶

Every hour, summarise the last hour's events. ~500k events. Processing is mostly aggregation in memory.

Tool: errgroup with SetLimit(NumCPU). Aggregation-heavy is CPU-bound, NumCPU is right.

func summarise(ctx context.Context, events []Event) (Summary, error) {
    type partialResult struct{ idx int; partial Summary }
    out := make(chan partialResult, runtime.NumCPU())
    g, ctx := errgroup.WithContext(ctx)
    g.SetLimit(runtime.NumCPU())
    chunks := chunkEvents(events, runtime.NumCPU())
    for i, chunk := range chunks {
        i, chunk := i, chunk
        g.Go(func() error {
            s, err := summariseChunk(chunk)
            if err != nil { return err }
            out <- partialResult{i, s}
            return nil
        })
    }
    go func() { g.Wait(); close(out) }()

    var combined Summary
    for r := range out {
        combined = merge(combined, r.partial)
    }
    return combined, g.Wait()
}

Map-reduce pattern. errgroup is enough. Pool unnecessary.

Workload S17.7: Service mesh sidecar¶

A high-throughput sidecar handling 100k req/sec, each requiring a small bit of routing logic (~50 μs).

Constraints: - 4 cores per pod. - Latency budget: 1ms p99.

Analysis: - 100k req/sec × 0.00005s = 5 active CPU-equivalent. 4 cores can handle it. - Spawn rate 100k/sec — high enough that goroutine creation matters.

Tool: ants with K=1024. Lower per-Submit overhead than errgroup at this rate.

But wait — for 50μs of work, goroutine spawn (1-2μs) is 2-4% overhead. ants Submit is ~250ns, which is 0.5%. Difference ~3% CPU.

That's measurable. Use ants.

pool, _ := ants.NewPool(1024)
for req := range incoming {
    req := req
    pool.Submit(func() { route(req) })
}