Unlimited Goroutines — Senior Level¶

Table of Contents¶

1. Introduction
2. The Senior Mindset: Concurrency as a Resource
3. A Taxonomy of Bounded-Fan-Out Patterns
4. Backpressure as the Primary Cure
5. Semaphores in Depth
6. golang.org/x/sync/semaphore Internals and Usage
7. errgroup.SetLimit and the Group Pattern
8. Bounded Worker Pools at Scale
9. Request-Scoped Concurrency Budgets
10. Multi-Tier Bounding
11. Per-Tenant and Per-Resource Quotas
12. Admission Control and Queue Management
13. Goroutine Leak Detection with goleak
14. The Bounded Pipeline Anti-Pattern Test Harness
15. Production Case Studies
16. Failure Modes Beyond OOM
17. Designing for Graceful Degradation
18. The Refactor Playbook
19. Cross-Service Concurrency Contracts
20. Self-Assessment
21. Summary

Introduction¶

At junior level the lesson was "do not write for ... { go ... } without a bound." At middle level the lesson was "choose the bound from a model — CPU count, downstream capacity, in-flight memory budget — and implement it with a worker pool, semaphore, or errgroup.SetLimit." At senior level the question moves up one layer: how do you design a system in which the bound is a first-class architectural property, propagated through every layer, observable in production, and verified by tests that will catch a regression months from now?

A senior Go engineer is rarely the person who writes the first for ... go in a codebase. They are the person who answers, six months later, "why did pod-3 OOM at 03:17 UTC while pods 1, 2, 4, 5 were healthy?" and discovers that one tenant on pod-3 triggered a code path with an unbounded range over their own catalogue. The fan-out itself was added to the codebase a year ago by an engineer who had only worked with bounded inputs. The bound was implicit in the dataset; nothing in the type system, in the lint configuration, or in the tests prevented an unbounded input from arriving.

This document is about how to make the bound explicit. It covers:

• The patterns that turn an unbounded loop into a bounded one (semaphore, errgroup with limit, worker pool, batch dispatcher).
• The library APIs you should know to a level where you can read their source: golang.org/x/sync/semaphore, golang.org/x/sync/errgroup, sync.Cond for hand-rolled cases.
• The practice of attributing a concurrency budget to a request, a tenant, a resource, or a downstream service.
• Leak detection (go.uber.org/goleak) and how to integrate it into your CI.
• A small library of production case studies — what failed, why, what the fix was, and what the team measured after.

By the end you should be able to design a service that is provably bounded: someone can pull a number out of the design and say "we will never have more than 4096 in-flight requests in this process, because here is the chain of Acquire/SetLimit/make(chan, N) calls that enforce it."

The Senior Mindset: Concurrency as a Resource¶

The most damaging mental model for a senior engineer is "goroutines are cheap, so use as many as you need." This was true in 2012 and is still cited in talks and documentation. It is misleading in 2026 for three reasons:

1. The goroutine is the cheap part. The work it does is not. A goroutine that issues a database query holds a connection. A goroutine that uploads to S3 holds a TCP socket and a TLS buffer. A goroutine that calls a downstream service contributes to that service's load. The bound you care about is rarely "how many goroutines can the runtime hold?" — it is "how many resources of type X can my service safely hold open simultaneously?"
2. Aggregate cost is non-linear. Ten thousand idle goroutines cost a few tens of megabytes. Ten thousand goroutines each contending for the same sync.Mutex push the scheduler into a degenerate state where every wake-up triggers a re-acquire and the throughput collapses. The runtime is engineered for high goroutine count, not for arbitrarily high contention. Contention scales with the square of contenders in the worst case.
3. The blast radius matters more than the average. A service that comfortably runs at 5000 goroutines for an hour, then briefly spikes to 500 000 because a single user uploaded a large CSV, is a service whose 99.99-percentile is measured by the spike, not by the average. The bound is what determines whether the spike is a small latency bump or a pod restart that triggers a partial outage.

The senior mindset is the opposite of "goroutines are cheap." It is "concurrency is a finite resource, the size of which I am responsible for choosing and defending."

A short vocabulary¶

This vocabulary will recur throughout the document. The distinction between "bound" and "budget" matters: a bound is a hard ceiling enforced by a primitive (semaphore, channel buffer); a budget is the allocation of that ceiling across logical scopes (per-tenant, per-handler).

How the senior thinks about every go statement¶

Before you write go, mentally check:

1. Who is calling this code path? A library function called from an unknown number of goroutines must not itself add unbounded fan-out, because the fan-out compounds. A leaf function in a request handler with a bounded input can be more relaxed.
2. What input drives the count? If the count is a function of user-supplied data (slice length, JSON array size, line count of an upload), the bound is your responsibility, not the user's. If the count is a function of a known-finite resource (number of CPU cores, number of shards in your own database), the bound exists implicitly and you only need to make it explicit in a comment or assertion.
3. What happens at 10x the worst case? Imagine the worst input you have seen multiplied by ten. Will the system absorb it (good), shed load gracefully (good), queue indefinitely (bad), or crash (worst)?
4. What is the resource downstream? Does each goroutine consume a database connection from a pool of 50? An OS file descriptor from a ulimit of 65 535? A slot in a remote API that returns 429 above 100/s? The bound on goroutines should be derived from the smallest downstream resource.
5. Is the parent's lifetime bounded? The goroutine cannot outlive the parent if the parent waits for it. If the parent is a request handler with a 30-second timeout, the goroutine inherits that bound. If the parent is main, the goroutine can live forever.

Five questions, every spawn site. After a few months it becomes reflexive.

A Taxonomy of Bounded-Fan-Out Patterns¶

There are four families of bounded-fan-out pattern in idiomatic Go. Knowing all four — and when each is appropriate — is part of the senior toolkit.

Family 1: The fixed worker pool¶

package pool

import (
    "context"
    "sync"
)

type Job func(ctx context.Context) error

type Pool struct {
    workers int
    jobs    chan Job
    wg      sync.WaitGroup
}

func New(workers, queue int) *Pool {
    p := &Pool{
        workers: workers,
        jobs:    make(chan Job, queue),
    }
    return p
}

func (p *Pool) Start(ctx context.Context) {
    for i := 0; i < p.workers; i++ {
        p.wg.Add(1)
        go p.worker(ctx)
    }
}

func (p *Pool) worker(ctx context.Context) {
    defer p.wg.Done()
    for {
        select {
        case <-ctx.Done():
            return
        case job, ok := <-p.jobs:
            if !ok {
                return
            }
            _ = job(ctx)
        }
    }
}

func (p *Pool) Submit(ctx context.Context, j Job) error {
    select {
    case <-ctx.Done():
        return ctx.Err()
    case p.jobs <- j:
        return nil
    }
}

func (p *Pool) Stop() {
    close(p.jobs)
    p.wg.Wait()
}

Use when: the pool is long-lived (process-lifetime), jobs are uniform, and you want backpressure to bubble up to callers naturally via Submit blocking.

Trade-offs: requires explicit lifecycle (Start and Stop), errors must be carried in the Job closure (or via a separate result channel), and a slow job parks a worker.

Family 2: The per-call semaphore¶

package fanout

import (
    "context"

    "golang.org/x/sync/semaphore"
)

func ProcessAll(ctx context.Context, items []Item, fn func(context.Context, Item) error) error {
    sem := semaphore.NewWeighted(64)
    errCh := make(chan error, len(items))

    for _, it := range items {
        if err := sem.Acquire(ctx, 1); err != nil {
            return err
        }
        go func(it Item) {
            defer sem.Release(1)
            if err := fn(ctx, it); err != nil {
                errCh <- err
            }
        }(it)
    }

    if err := sem.Acquire(ctx, 64); err != nil {
        return err
    }
    close(errCh)
    var firstErr error
    for err := range errCh {
        if firstErr == nil {
            firstErr = err
        }
    }
    return firstErr
}

Use when: the fan-out is per-call (one batch, one set of inputs, finite duration), and you want a simple inline pattern without the ceremony of Start/Stop.

Trade-offs: error handling is manual (you must wire your own channel); the loop body is more verbose than errgroup.Go; care is required to release on every exit path (panic, ctx cancel, early return).

Family 3: errgroup.WithContext plus SetLimit¶

package fanout

import (
    "context"

    "golang.org/x/sync/errgroup"
)

func ProcessAll(ctx context.Context, items []Item, fn func(context.Context, Item) error) error {
    g, gctx := errgroup.WithContext(ctx)
    g.SetLimit(64)

    for _, it := range items {
        it := it
        g.Go(func() error {
            return fn(gctx, it)
        })
    }
    return g.Wait()
}

Use when: this is the default for per-call fan-out in modern Go (1.20+). It composes context cancellation, error aggregation, and a concurrency limit into a single primitive. There is rarely a good reason to write Family 2 instead, unless you need fine-grained Acquire/Release semantics (e.g. weighted tokens).

Trade-offs: SetLimit does not provide weighted tokens (every job costs 1). For weighted jobs, fall back to semaphore. The implementation queues Go calls when at the limit, which means g.Go blocks the caller; this is usually what you want (it produces backpressure into the loop), but be aware of it.

Family 4: The pipeline of bounded stages¶

package pipeline

import (
    "context"
    "sync"
)

type Stage[In, Out any] struct {
    Workers int
    Fn      func(context.Context, In) (Out, error)
}

func Run[In, Out any](ctx context.Context, in <-chan In, s Stage[In, Out]) <-chan Out {
    out := make(chan Out, s.Workers)
    var wg sync.WaitGroup
    for i := 0; i < s.Workers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for v := range in {
                r, err := s.Fn(ctx, v)
                if err != nil {
                    continue
                }
                select {
                case <-ctx.Done():
                    return
                case out <- r:
                }
            }
        }()
    }
    go func() {
        wg.Wait()
        close(out)
    }()
    return out
}

Use when: the work is naturally a chain (parse → transform → write), each stage has its own concurrency, and you want the upstream to apply backpressure to the downstream simply by the buffered channel filling up.

Trade-offs: pipelines hide structure inside channels and require careful close-cascade discipline; error handling is awkward unless you wrap the value type with a result struct; complex topologies (fan-out from one stage into N parallel downstreams) need careful design.

Choosing among the four¶

In a typical microservice you will mostly use Family 3 inside request handlers and Family 1 for background workers. Families 2 and 4 are reserved for specialised cases.

Backpressure as the Primary Cure¶

The defining property of a bounded fan-out is not that "at most N goroutines exist at once." It is that "if work arrives faster than the system can process it, the producer is slowed or told to stop, rather than the system silently growing without bound."

Backpressure is the signal by which the bound is communicated to the producer. Without backpressure, a bound is just a queue size — when the queue fills, you have to decide what happens, and most of the time the right answer is "the producer waits."

Backpressure shapes¶

There are three shapes of backpressure in Go:

1. Blocking send on a buffered channel. The producer's ch <- v blocks when the buffer is full. The producer is naturally stalled.
2. Blocking Acquire on a semaphore. The producer's sem.Acquire(ctx, 1) blocks when no tokens are available. Equivalent to (1).
3. Returning a "try later" error. The producer's Submit returns immediately with ErrFull, and the caller must decide whether to retry, drop, or fail.

Shapes (1) and (2) are implicit backpressure: the caller does not need to know what to do; they just wait. Shape (3) is explicit backpressure: the caller must implement a retry policy, a drop policy, or a fail policy.

In a request-handling context, you almost always want shape (3) at the outermost boundary (the HTTP handler), because the client will time out anyway. You almost always want shape (1) or (2) at inner boundaries (between worker pools), because the producer is also a goroutine and waiting is cheap.

A worked example: end-to-end backpressure in a CSV importer¶

Imagine a service that imports CSV files into a database. The flow:

HTTP POST → parse CSV → enrich each row → write to DB

If a user uploads a 10 GB CSV, the naive flow for _, row := range rows { go enrich(row); go write(row) } will fan out 100 million goroutines, exhaust memory, exhaust the DB connection pool, and crash. The bounded version:

package importer

import (
    "context"
    "encoding/csv"
    "io"
    "net/http"

    "golang.org/x/sync/errgroup"
)

type Service struct {
    EnrichPool int // e.g. 32
    DBPool     int // e.g. 16 (matches DB connection pool size)
    BufferRows int // e.g. 256
}

func (s *Service) Import(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    rows := make(chan []string, s.BufferRows)
    enriched := make(chan EnrichedRow, s.BufferRows)

    g, gctx := errgroup.WithContext(ctx)

    g.Go(func() error {
        defer close(rows)
        cr := csv.NewReader(r.Body)
        for {
            row, err := cr.Read()
            if err == io.EOF {
                return nil
            }
            if err != nil {
                return err
            }
            select {
            case <-gctx.Done():
                return gctx.Err()
            case rows <- row:
            }
        }
    })

    enrichG, enrichCtx := errgroup.WithContext(gctx)
    enrichG.SetLimit(s.EnrichPool)
    g.Go(func() error {
        defer close(enriched)
        for row := range rows {
            row := row
            enrichG.Go(func() error {
                e, err := enrich(enrichCtx, row)
                if err != nil {
                    return err
                }
                select {
                case <-enrichCtx.Done():
                    return enrichCtx.Err()
                case enriched <- e:
                }
                return nil
            })
        }
        return enrichG.Wait()
    })

    writeG, writeCtx := errgroup.WithContext(gctx)
    writeG.SetLimit(s.DBPool)
    g.Go(func() error {
        for e := range enriched {
            e := e
            writeG.Go(func() error {
                return writeRow(writeCtx, e)
            })
        }
        return writeG.Wait()
    })

    if err := g.Wait(); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    w.WriteHeader(http.StatusOK)
}

type EnrichedRow struct {
    ID    string
    Value string
}

func enrich(ctx context.Context, row []string) (EnrichedRow, error) { return EnrichedRow{}, nil }
func writeRow(ctx context.Context, e EnrichedRow) error             { return nil }

Trace the backpressure:

• If writeRow is slow, writeG.Go blocks (it's at limit), so the loop reading enriched stalls.
• If the enriched channel buffer fills, the enrich goroutine's send blocks.
• If enrichG.Go is at limit, the loop reading rows stalls.
• If the rows channel buffer fills, the CSV reader's send blocks.
• The CSV reader's r.Body.Read stops being called.
• TCP backpressure flows back to the client, who sees their POST slow down.

The client's upload speed is now governed by the database's write speed, with bounded memory in between. This is what end-to-end backpressure looks like.

If r.Context() is cancelled (client disconnect), every stage sees gctx.Done(), closes its channels, and returns. No goroutine leaks.

When to drop instead of block¶

End-to-end backpressure is the right default. There are two situations where dropping is better:

1. Stale-tolerant work. If the work is "send a real-time price update to a websocket client," and a new update has arrived while you were waiting to send the previous one, dropping the previous update is correct. The consumer wants freshness, not completeness.
2. Unfair producers. If one producer is faster than the rest and you don't want them to monopolise the pipeline, drop their excess and serve everyone.

The drop policy must be explicit:

select {
case ch <- v:
case <-ctx.Done():
    return ctx.Err()
default:
    droppedCounter.Inc()
}

Note the default: without it, the select blocks. With it, the send is non-blocking and the drop is counted.

Semaphores in Depth¶

A semaphore is a counter that supports two operations:

• Acquire(n): block until count >= n, then count -= n.
• Release(n): count += n, wake any waiters whose n is now available.

A binary semaphore (n = 1) is essentially a mutex. A counting semaphore with capacity C is the canonical way to limit concurrency to at most C.

Counting vs weighted¶

The simplest semaphore counts indistinguishable tokens. Every Acquire(1) takes one token; every Release(1) returns one. This is the case for "at most 64 in-flight HTTP requests."

A weighted semaphore counts variable-size tokens. Acquire(8) reserves 8 units of capacity; Acquire(1) reserves 1. This is useful when jobs have different "cost":

• A small task costs 1 unit; a large task costs 4. A capacity-of-32 semaphore admits 32 small or 8 large or any mix.
• A small HTTP request costs 1; a large request costs 16 (proportional to memory footprint).

golang.org/x/sync/semaphore is weighted. errgroup.SetLimit is counting only.

Hand-rolled counting semaphore¶

The simplest implementation in Go is a buffered channel:

type Sem struct{ ch chan struct{} }

func NewSem(n int) *Sem { return &Sem{ch: make(chan struct{}, n)} }
func (s *Sem) Acquire() { s.ch <- struct{}{} }
func (s *Sem) Release() { <-s.ch }

A chan struct{} with capacity N is a perfectly correct counting semaphore. Acquire blocks when N elements are already in the channel; Release un-blocks one waiter. The implementation is two lines and is what you should use when you don't need weighted tokens, context cancellation on Acquire, or fairness guarantees.

Adding context cancellation:

func (s *Sem) Acquire(ctx context.Context) error {
    select {
    case <-ctx.Done():
        return ctx.Err()
    case s.ch <- struct{}{}:
        return nil
    }
}

Now Acquire returns an error when the context is cancelled — useful in handlers that must abandon work when the client disconnects.

Fairness¶

A chan struct{} semaphore is FIFO among waiters: the first goroutine to block on ch <- struct{}{} is the first to be admitted when a Release happens. The Go channel implementation maintains a FIFO queue of waiters per direction.

If you do not need FIFO fairness — for example, if all waiters are equivalent and "first wins" is fine — the channel implementation is more than sufficient. If you need priority (one waiter type should be admitted before another), the channel is not enough; you need a custom implementation with a priority queue.

When to choose the standard library Mutex instead¶

A sync.Mutex is a binary semaphore. If your bound is 1 (one in-flight at a time), use Mutex. Reasons:

• It is more familiar to readers.
• It plays well with the race detector.
• It does not allocate a channel.

For any bound greater than 1, a counting semaphore (channel or x/sync/semaphore) is correct; Mutex is not.

golang.org/x/sync/semaphore Internals and Usage¶

The standard counting/weighted semaphore in Go is golang.org/x/sync/semaphore. The package is small enough that every senior Go engineer should be able to read it and explain how it works.

The public API¶

package semaphore

func NewWeighted(n int64) *Weighted

type Weighted struct {
    // unexported
}

func (s *Weighted) Acquire(ctx context.Context, n int64) error
func (s *Weighted) TryAcquire(n int64) bool
func (s *Weighted) Release(n int64)

NewWeighted(N) creates a semaphore of total capacity N. Acquire(ctx, n) waits until n units are available, then takes them; it returns ctx.Err() if the context is cancelled before acquisition. TryAcquire(n) is the non-blocking version: it returns true if it acquired, false otherwise. Release(n) returns n units, possibly waking waiters.

How it works¶

The implementation maintains:

• size: the total capacity (immutable after construction).
• cur: the currently allocated tokens.
• waiters: a FIFO list of waiters, each described by {n: requested count, ready: channel to signal}.
• mu: a mutex protecting cur and waiters.

Acquire(ctx, n) algorithm:

1. Lock mu.
2. If cur + n <= size and waiters is empty (or only later-arrived waiters), increment cur by n, unlock, return nil.
3. Otherwise create a waiter with a ready channel, append to waiters, unlock.
4. Block on select between ctx.Done() and ready.
5. If ctx.Done() fires first, remove yourself from waiters and try to consume your slot (in case Release was racing).
6. Return.

Release(n) algorithm:

1. Lock mu.
2. Decrement cur by n.
3. While waiters is non-empty and the head waiter's n fits in the remaining capacity: pop them, increment cur by their n, signal their ready.
4. Unlock.

Two properties to note:

• FIFO fairness. A waiter who requested 4 cannot be passed over by a later waiter who requested 1, even if 1 fits and 4 does not. The package deliberately blocks small later waiters behind large earlier waiters to prevent starvation. This is sometimes the wrong policy (you may prefer "fill the slot if you can"), but it is a deliberate choice.
• No starvation guarantee for large weights. If you request n equal to or larger than size, you will block until every current holder has released; you can be starved indefinitely if releases are bursty. In practice, if you find yourself acquiring close to size, your design is probably wrong — the weight system is meant for fractional reservations.

A real Acquire/Release with ctx cancellation¶

package crawler

import (
    "context"
    "net/http"

    "golang.org/x/sync/semaphore"
)

type Crawler struct {
    sem    *semaphore.Weighted
    client *http.Client
}

func New(maxInFlight int64) *Crawler {
    return &Crawler{
        sem:    semaphore.NewWeighted(maxInFlight),
        client: &http.Client{},
    }
}

func (c *Crawler) Fetch(ctx context.Context, url string) (*http.Response, error) {
    if err := c.sem.Acquire(ctx, 1); err != nil {
        return nil, err
    }
    defer c.sem.Release(1)

    req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
    if err != nil {
        return nil, err
    }
    return c.client.Do(req)
}

This is what a per-call concurrency limit looks like in production. The semaphore is owned by the Crawler and shared across goroutines. Each call to Fetch waits for a slot.

Note three details:

1. The Acquire uses ctx, so a cancelled request does not consume a slot it cannot use.
2. The defer Release(1) matches the Acquire(ctx, 1). If you Acquire(ctx, 8), you must Release(8). Mismatches are catastrophic — releasing more than you acquired drives cur negative (the package does not check) and corrupts the semaphore.
3. The Release happens whether the HTTP call succeeded or failed.

Weighted use case: memory-budgeted decoder¶

package decoder

import (
    "context"
    "fmt"

    "golang.org/x/sync/semaphore"
)

type Pool struct {
    memBudget *semaphore.Weighted
}

func NewPool(memMB int64) *Pool {
    return &Pool{memBudget: semaphore.NewWeighted(memMB)}
}

func (p *Pool) Decode(ctx context.Context, sizeMB int64, data []byte) ([]byte, error) {
    if sizeMB > 256 {
        return nil, fmt.Errorf("image too large: %d MB", sizeMB)
    }
    if err := p.memBudget.Acquire(ctx, sizeMB); err != nil {
        return nil, err
    }
    defer p.memBudget.Release(sizeMB)
    return decodeImpl(data, sizeMB)
}

func decodeImpl(data []byte, sizeMB int64) ([]byte, error) { return nil, nil }

The pool has a total memory budget. A small image consumes 1 MB; a large image consumes 256 MB. The semaphore allows many small concurrent decodes or few large ones, with the total memory footprint bounded. This is the canonical use case for the weighted semaphore.

Common mistakes¶

Mistake 1: Release-before-Acquire on the error path.

if err := sem.Acquire(ctx, 1); err != nil {
    sem.Release(1) // BUG — never acquired
    return err
}

The Release corrupts the semaphore. The fix is the defer idiom: acquire first, then defer Release(1), then do the work.

Mistake 2: Holding the semaphore across the wrong scope.

sem.Acquire(ctx, 1)
result, err := doSlowWork()
if cacheUnsafe(result) {
    return err
}
sem.Release(1)
// ...

If the early return path forgets to release, the semaphore leaks. Always defer Release(1) immediately after Acquire. Restructure the code to avoid early returns between Acquire and Release.

Mistake 3: Mixing weighted and counting use.

If one caller does Acquire(1) and another does Acquire(8) on the same semaphore, with no clear semantics for what the weights mean, you have an underspecified system. Document the meaning of the weight (1 unit = 1 MB, or 1 unit = 1 connection) and stick to it.

Mistake 4: TryAcquire without a retry plan.

if !sem.TryAcquire(1) {
    return errors.New("busy")
}

This is fine if "busy" is a meaningful response to the caller. It is not fine if the caller has no way to handle "busy" other than to retry in a tight loop — that is just a spin lock with extra steps. Prefer Acquire with a context deadline.

errgroup.SetLimit and the Group Pattern¶

golang.org/x/sync/errgroup is the workhorse of bounded per-call fan-out. It was already a great primitive when it had only Go and Wait. The addition of SetLimit in 2022 made it the default choice.

The API¶

package errgroup

type Group struct {
    // unexported
}

func WithContext(ctx context.Context) (*Group, context.Context)

func (g *Group) Go(f func() error)
func (g *Group) TryGo(f func() error) bool
func (g *Group) SetLimit(n int)
func (g *Group) Wait() error

Two constructors: the zero value Group{} is usable but has no shared cancellation context; WithContext creates a derived context that is cancelled when any Go returns a non-nil error or Wait returns. Use WithContext 99% of the time.

SetLimit(n) configures the group to allow at most n concurrent goroutines. If n is negative, there is no limit (the default). Calling SetLimit after any Go has been called panics — you must set the limit before the first Go.

Go(f) enqueues f for execution. If the group is at the limit, Go blocks until a slot is free. TryGo(f) is the non-blocking version: it returns true if it started f, false if it would have blocked.

Wait() blocks until all Go'd functions have returned, then returns the first non-nil error (if any).

Worked example¶

package fetcher

import (
    "context"
    "io"
    "net/http"

    "golang.org/x/sync/errgroup"
)

type Fetcher struct {
    Client       *http.Client
    Concurrency  int
}

func (f *Fetcher) FetchAll(ctx context.Context, urls []string) (map[string][]byte, error) {
    g, gctx := errgroup.WithContext(ctx)
    g.SetLimit(f.Concurrency)

    results := make(map[string][]byte, len(urls))
    var mu sync.Mutex

    for _, u := range urls {
        u := u
        g.Go(func() error {
            req, err := http.NewRequestWithContext(gctx, http.MethodGet, u, nil)
            if err != nil {
                return err
            }
            resp, err := f.Client.Do(req)
            if err != nil {
                return err
            }
            defer resp.Body.Close()
            body, err := io.ReadAll(resp.Body)
            if err != nil {
                return err
            }
            mu.Lock()
            results[u] = body
            mu.Unlock()
            return nil
        })
    }

    if err := g.Wait(); err != nil {
        return nil, err
    }
    return results, nil
}

// somewhere
import "sync"
var _ sync.Mutex

Note three patterns:

1. The u := u shadow. Pre-Go 1.22 this is required to capture the per-iteration value. On Go 1.22+ the loop variable is per-iteration by default, but the explicit shadow remains a clear signal and is portable.
2. The gctx is what the goroutines use. If any Go returns an error, gctx is cancelled, which signals every other goroutine to abandon their HTTP request.
3. Shared state needs its own synchronisation. The errgroup does not synchronise access to the results map. We add a sync.Mutex.

Go blocks when at limit — why this matters¶

Consider:

g, gctx := errgroup.WithContext(ctx)
g.SetLimit(4)
for _, u := range hugeList {
    u := u
    g.Go(func() error { return slowFetch(gctx, u) })
}
return g.Wait()

If hugeList has 10 million entries and slowFetch takes 1 second, the for loop will not pile up 10 million enqueued goroutines. It will execute 4 goroutines, then g.Go will block on the 5th, then proceed when one finishes. The "fan-out" is, in fact, "fan in slow motion."

This is the desired behaviour, but it has a subtle implication: the loop's iteration speed is governed by the worker speed. If you wanted to also do other work in the loop (e.g. log progress), you can:

for i, u := range hugeList {
    u := u
    if i%100 == 0 {
        log.Printf("submitted %d / %d", i, len(hugeList))
    }
    g.Go(func() error { return slowFetch(gctx, u) })
}

The log line prints every 100 successful submissions, which is approximately every 100 / 4 = 25 seconds of wall time — useful for monitoring.

When TryGo is the right choice¶

TryGo is rarely the right choice. The reason Go's blocking behaviour exists is that it produces backpressure: the loop slows down to match the workers. TryGo lets the loop run ahead at full speed, dropping work that does not fit. This is correct only when "drop" is acceptable.

A legitimate TryGo use case: a periodic health check that should never queue up.

for {
    select {
    case <-tick.C:
        if !g.TryGo(healthCheck) {
            log.Println("skipping health check: still running previous")
        }
    case <-ctx.Done():
        return
    }
}

If a health check takes longer than the tick interval, we skip the next one rather than queue it. The metric we care about is "is the system healthy now," not "has every scheduled check ever run."

Pitfall: SetLimit does not bound Wait()-related buffering¶

errgroup.SetLimit(n) limits concurrent execution, not the in-flight queue. There is no queue — Go blocks the caller. This is different from a worker pool, where Submit puts items in a queue.

If you want a queue (so producer can move on while workers process), you have to add one yourself:

type bounded struct {
    g    *errgroup.Group
    gctx context.Context
    in   chan func() error
}

func newBounded(ctx context.Context, concurrency, queue int) *bounded {
    g, gctx := errgroup.WithContext(ctx)
    g.SetLimit(concurrency)
    b := &bounded{g: g, gctx: gctx, in: make(chan func() error, queue)}
    g.Go(func() error {
        for fn := range b.in {
            fn := fn
            g.Go(fn)
        }
        return nil
    })
    return b
}

But this hides backpressure inside the buffered channel and adds complexity. Prefer plain Go (which produces backpressure naturally) unless you have a documented reason for a queue.

Bounded Worker Pools at Scale¶

For long-lived services, the worker pool is the dominant pattern. A worker pool has three components:

1. An input channel for jobs.
2. A fixed set of goroutines that read from the input channel.
3. A lifecycle controller (Start, Stop, Wait).

The hard part is not the basic structure; it is making the pool production-grade: cleanly stoppable, observable, robust to panics, and well-behaved under load.

A production worker pool¶

package pool

import (
    "context"
    "errors"
    "fmt"
    "log"
    "runtime/debug"
    "sync"
    "sync/atomic"
    "time"
)

type Job interface {
    ID() string
    Execute(ctx context.Context) error
}

type Pool struct {
    workers     int
    queue       chan Job
    sem         chan struct{}
    inflight    int64
    submitted   int64
    completed   int64
    failed      int64
    panicked    int64

    started     atomic.Bool
    stopOnce    sync.Once
    stopped     chan struct{}
    workersWG   sync.WaitGroup
    ctx         context.Context
    cancel      context.CancelFunc

    onPanic     func(jobID string, panicValue interface{})
    onComplete  func(jobID string, dur time.Duration, err error)
}

type Config struct {
    Workers    int
    QueueDepth int
    OnPanic    func(jobID string, panicValue interface{})
    OnComplete func(jobID string, dur time.Duration, err error)
}

func New(cfg Config) *Pool {
    if cfg.Workers <= 0 {
        cfg.Workers = 1
    }
    if cfg.QueueDepth < 0 {
        cfg.QueueDepth = 0
    }
    if cfg.OnPanic == nil {
        cfg.OnPanic = func(id string, v interface{}) {
            log.Printf("pool: job %s panicked: %v\n%s", id, v, debug.Stack())
        }
    }
    if cfg.OnComplete == nil {
        cfg.OnComplete = func(string, time.Duration, error) {}
    }
    return &Pool{
        workers:    cfg.Workers,
        queue:      make(chan Job, cfg.QueueDepth),
        sem:        make(chan struct{}, cfg.Workers),
        stopped:    make(chan struct{}),
        onPanic:    cfg.OnPanic,
        onComplete: cfg.OnComplete,
    }
}

func (p *Pool) Start(ctx context.Context) {
    if !p.started.CompareAndSwap(false, true) {
        return
    }
    p.ctx, p.cancel = context.WithCancel(ctx)
    for i := 0; i < p.workers; i++ {
        p.workersWG.Add(1)
        go p.worker(i)
    }
}

func (p *Pool) worker(id int) {
    defer p.workersWG.Done()
    for {
        select {
        case <-p.ctx.Done():
            return
        case job, ok := <-p.queue:
            if !ok {
                return
            }
            p.run(job)
        }
    }
}

func (p *Pool) run(job Job) {
    atomic.AddInt64(&p.inflight, 1)
    defer atomic.AddInt64(&p.inflight, -1)
    start := time.Now()
    var err error
    func() {
        defer func() {
            if r := recover(); r != nil {
                atomic.AddInt64(&p.panicked, 1)
                p.onPanic(job.ID(), r)
                err = fmt.Errorf("panic: %v", r)
            }
        }()
        err = job.Execute(p.ctx)
    }()
    if err != nil {
        atomic.AddInt64(&p.failed, 1)
    }
    atomic.AddInt64(&p.completed, 1)
    p.onComplete(job.ID(), time.Since(start), err)
}

func (p *Pool) Submit(ctx context.Context, job Job) error {
    if !p.started.Load() {
        return errors.New("pool not started")
    }
    atomic.AddInt64(&p.submitted, 1)
    select {
    case <-ctx.Done():
        return ctx.Err()
    case <-p.ctx.Done():
        return errors.New("pool stopped")
    case p.queue <- job:
        return nil
    }
}

func (p *Pool) TrySubmit(job Job) bool {
    if !p.started.Load() {
        return false
    }
    select {
    case p.queue <- job:
        atomic.AddInt64(&p.submitted, 1)
        return true
    default:
        return false
    }
}

func (p *Pool) Stop(ctx context.Context) error {
    var stopErr error
    p.stopOnce.Do(func() {
        close(p.queue)
        done := make(chan struct{})
        go func() {
            p.workersWG.Wait()
            close(done)
        }()
        select {
        case <-done:
            // workers exited cleanly
        case <-ctx.Done():
            stopErr = ctx.Err()
            p.cancel()
            <-done
        }
        close(p.stopped)
    })
    return stopErr
}

func (p *Pool) Stats() (submitted, completed, failed, panicked, inflight int64) {
    return atomic.LoadInt64(&p.submitted),
        atomic.LoadInt64(&p.completed),
        atomic.LoadInt64(&p.failed),
        atomic.LoadInt64(&p.panicked),
        atomic.LoadInt64(&p.inflight)
}

This pool is approximately 130 lines and demonstrates the production-grade features:

• Idempotent Start and Stop via atomic.Bool and sync.Once.
• Graceful shutdown with timeout: Stop(ctx) waits for workers but if ctx expires it cancels the workers' context and waits anyway.
• Panic recovery per job, with an injectable handler.
• Atomic counters for metrics, exposed via Stats.
• Backpressure: Submit blocks (via the channel send) when the queue is full and the context is alive.
• TrySubmit for drop-on-overflow callers.

Sizing the pool — a senior-level approach¶

The naive answer is runtime.NumCPU(). The senior answer is "measure."

Step 1: Identify the bottleneck¶

For each pool, ask:

• Is the job CPU-bound? If so, pool size = NumCPU() to 2 × NumCPU().
• Is the job I/O-bound? If so, pool size = downstream concurrency limit.
• Is the job memory-bound? Pool size = budget / per-job-memory.
• Is the job mixed? Use Little's Law: throughput × latency = in-flight. Pick a target throughput and known latency; the product is the pool size.

Step 2: Validate with a benchmark¶

package pool_test

import (
    "context"
    "fmt"
    "testing"
    "time"

    "yourpkg/pool"
)

type fakeJob struct {
    id  string
    dur time.Duration
}

func (f fakeJob) ID() string                          { return f.id }
func (f fakeJob) Execute(_ context.Context) error     { time.Sleep(f.dur); return nil }

func BenchmarkPoolSizes(b *testing.B) {
    for _, n := range []int{1, 2, 4, 8, 16, 32, 64, 128, 256, 512} {
        b.Run(fmt.Sprintf("workers=%d", n), func(b *testing.B) {
            p := pool.New(pool.Config{Workers: n, QueueDepth: n * 4})
            p.Start(context.Background())
            defer func() { _ = p.Stop(context.Background()) }()
            b.ResetTimer()
            for i := 0; i < b.N; i++ {
                _ = p.Submit(context.Background(), fakeJob{id: "", dur: 10 * time.Millisecond})
            }
            b.StopTimer()
            // wait for drain
            for {
                _, c, _, _, inflight := p.Stats()
                if c == int64(b.N) && inflight == 0 {
                    break
                }
                time.Sleep(time.Millisecond)
            }
        })
    }
}

Run with go test -bench BenchmarkPoolSizes -benchtime=10s. Plot the throughput vs. pool size. You will see throughput rise, plateau, and (often) fall as contention dominates. The optimum is the plateau's left edge.

Step 3: Pin the value with a justification¶

In production code, never write Workers: 64 without a comment:

p := pool.New(pool.Config{
    // 64 workers chosen via load test on 2026-03-15.
    // Bottleneck: downstream image-resizer with capacity 64 req/s.
    // Above 64 we hit 429s; below 32 we leave throughput on the table.
    Workers:    64,
    QueueDepth: 256,
})

The comment is for the engineer who, two years from now, will be asked "why 64? Can we make it 128?" The answer should not require an archeological dig through git blame.

Request-Scoped Concurrency Budgets¶

A request-scoped concurrency budget is a bound applied for the duration of a single request. The simplest example: a request handler that fans out to 8 backends in parallel.

func (s *Service) Handle(ctx context.Context, req Request) (Response, error) {
    g, gctx := errgroup.WithContext(ctx)
    g.SetLimit(8)

    results := make([]BackendResult, len(s.backends))
    for i, b := range s.backends {
        i, b := i, b
        g.Go(func() error {
            r, err := b.Query(gctx, req)
            if err != nil {
                return err
            }
            results[i] = r
            return nil
        })
    }
    if err := g.Wait(); err != nil {
        return Response{}, err
    }
    return aggregate(results), nil
}

This is fine for a single backend list. The bound is 8 — meaning at most 8 goroutines spawned by this handler exist simultaneously.

The compound problem¶

Now imagine the service has 1000 concurrent requests, each fanning out to 8 backends. That is 8000 simultaneous backend calls — and 8000 goroutines, plus the 1000 handler goroutines. If your downstream has a connection pool of size 100, you have 80x oversubscribed it.

The per-request bound is not enough. You need a service-wide bound. Two ways to add it:

Option A: A shared semaphore¶

type Service struct {
    backends    []Backend
    backendSem  *semaphore.Weighted // shared, total capacity 100
}

func NewService(backends []Backend, totalBudget int64) *Service {
    return &Service{
        backends:   backends,
        backendSem: semaphore.NewWeighted(totalBudget),
    }
}

func (s *Service) Handle(ctx context.Context, req Request) (Response, error) {
    g, gctx := errgroup.WithContext(ctx)
    g.SetLimit(8)

    results := make([]BackendResult, len(s.backends))
    for i, b := range s.backends {
        i, b := i, b
        g.Go(func() error {
            if err := s.backendSem.Acquire(gctx, 1); err != nil {
                return err
            }
            defer s.backendSem.Release(1)
            r, err := b.Query(gctx, req)
            if err != nil {
                return err
            }
            results[i] = r
            return nil
        })
    }
    if err := g.Wait(); err != nil {
        return Response{}, err
    }
    return aggregate(results), nil
}

Now every backend call across every request acquires from the same semaphore. Total in-flight backend calls = at most 100.

Option B: Limit at the source — admission control¶

type Service struct {
    backends []Backend
    requests *semaphore.Weighted // max in-flight requests
}

func (s *Service) Handler(w http.ResponseWriter, r *http.Request) {
    if !s.requests.TryAcquire(1) {
        http.Error(w, "service overloaded", http.StatusServiceUnavailable)
        return
    }
    defer s.requests.Release(1)
    // ... proceed normally; each request fans out to 8, total 8 × N requests
}

If requests has capacity 12, the max fan-out is 8 × 12 = 96, comfortably within the 100-connection pool.

Option B is preferable when the policy is "shed load at the edge." Option A is preferable when the policy is "queue requests but limit downstream impact."

Allocating a budget across heterogeneous backends¶

What if the 8 backends have different capacities? Backend 1 can handle 10 concurrent calls; backend 2 only 4; backends 3–8 are cloud-scale and effectively unlimited.

You need a per-backend semaphore, not a shared one:

type Backend struct {
    Name     string
    Capacity int
    sem      *semaphore.Weighted
}

func NewBackend(name string, capacity int) *Backend {
    return &Backend{Name: name, Capacity: capacity, sem: semaphore.NewWeighted(int64(capacity))}
}

func (b *Backend) Query(ctx context.Context, req Request) (BackendResult, error) {
    if err := b.sem.Acquire(ctx, 1); err != nil {
        return BackendResult{}, err
    }
    defer b.sem.Release(1)
    return b.queryImpl(ctx, req)
}

func (b *Backend) queryImpl(ctx context.Context, req Request) (BackendResult, error) {
    return BackendResult{}, nil
}

Each backend's Query self-limits. The request-scoped errgroup limits per-request parallelism. Together they form a layered bound: per-request bound 8, per-backend bound { 10, 4, ∞, ∞, ∞, ∞, ∞, ∞ }.

Multi-Tier Bounding¶

Real services have many tiers, each with its own bound. A typical layout:

ingress (admission control: 200 concurrent requests)
  → request handler (8 in-flight backend calls)
    → backend client (per-backend cap: 4 to 10)
      → connection pool (50 connections)
        → downstream service (its own capacity)

Each tier's bound should be derived from the tier below. Working from the bottom:

1. The downstream service publishes "we can serve 100 req/s before degrading."
2. Your connection pool sized 50 with avg call duration 500ms means in-flight 50, throughput 100/s — matches.
3. The per-backend cap of 10 means at most 10 of your goroutines wait in the connection pool queue at once.
4. The 8-in-flight-per-request and 200 concurrent requests means 1600 max backend calls in flight. Backend cap of 10 means 160 across all backends — well within 50.
5. The 200 concurrent requests bound is set so memory at peak is below 1 GB.

The arithmetic ensures no tier can overwhelm the tier below it. This is the architecture review you do before launching a service.

Visualising the bound stack¶

For documentation purposes, draw a small ASCII table:

+----------------------+----------+---------------+
| Tier                 | Bound    | Justification |
+----------------------+----------+---------------+
| HTTP server          | 200      | mem budget    |
| Per-request fan-out  | 8        | response p99  |
| Per-backend          | 10       | downstream    |
| Connection pool      | 50       | DB capacity   |
| Downstream service   | 100 rps  | their SLO     |
+----------------------+----------+---------------+

This table belongs in your service's README or runbook. New engineers will read it; future you will reread it during incident response.

When tiers conflict¶

If tier A's bound implies more work than tier B can absorb, the system will exhibit head-of-line blocking at tier B. Symptoms:

• Goroutines pile up at tier B's semaphore (visible in pprof goroutine).
• Tier A's queue fills (visible in your metrics).
• Latency p99 climbs.

The fix is to lower tier A's bound or raise tier B's capacity. There is no other answer.

A worked sizing exercise¶

Service: PDF report generator. Each request renders one PDF using a headless Chrome subprocess (50 MB RAM, 2s wall time).

• Pod has 4 GB RAM. After overhead, 3 GB is available for PDFs.
• 3 GB / 50 MB = 60 concurrent renders maximum on this pod.
• Conservatively, target 80% utilisation: 48 concurrent renders.
• Per request, fan-out = 1 (one PDF per request).
• So admission control should allow at most 48 concurrent requests.

At 2s per render and 48 concurrent renders, throughput = 24 req/s per pod. If your peak is 200 req/s, you need 200/24 ≈ 9 pods, plus headroom for hot pods, so 12 pods.

This is the kind of arithmetic that should appear in a design doc. Numbers are wrong sometimes — but a wrong number is debuggable; a missing number is not.

Per-Tenant and Per-Resource Quotas¶

Multi-tenant services need to isolate one tenant's load from another's. The mechanism is per-tenant semaphores.

Per-tenant concurrency cap¶

type TenantPool struct {
    mu    sync.RWMutex
    sems  map[string]*semaphore.Weighted
    limit int64
}

func NewTenantPool(perTenantLimit int64) *TenantPool {
    return &TenantPool{
        sems:  make(map[string]*semaphore.Weighted),
        limit: perTenantLimit,
    }
}

func (tp *TenantPool) sem(tenant string) *semaphore.Weighted {
    tp.mu.RLock()
    s, ok := tp.sems[tenant]
    tp.mu.RUnlock()
    if ok {
        return s
    }
    tp.mu.Lock()
    defer tp.mu.Unlock()
    if s, ok = tp.sems[tenant]; ok {
        return s
    }
    s = semaphore.NewWeighted(tp.limit)
    tp.sems[tenant] = s
    return s
}

func (tp *TenantPool) Acquire(ctx context.Context, tenant string) error {
    return tp.sem(tenant).Acquire(ctx, 1)
}

func (tp *TenantPool) Release(tenant string) {
    tp.sem(tenant).Release(1)
}