Wait-for-Empty-Channel — Middle Level¶

Table of Contents¶

1. Introduction
2. Recap of the Anti-Pattern
3. The Formal Race: Step by Step
4. Memory Model Implications
5. Why the Race Detector Sometimes Stays Quiet
6. Polling Versus Event-Driven Synchronisation
7. Correct Pattern Catalogue: WaitGroup
8. Correct Pattern Catalogue: Done Channel
9. Correct Pattern Catalogue: Context
10. Correct Pattern Catalogue: errgroup
11. Correct Pattern Catalogue: Close-and-Range
12. Choosing the Right Primitive
13. Composing Primitives: Real-World Layouts
14. WaitGroup Pitfalls You Will Hit
15. Done Channel Pitfalls You Will Hit
16. Context Pitfalls You Will Hit
17. Fan-Out Fan-In Without len
18. Worker Pools Without len
19. Pipelines Without len
20. Case Study: Order Ingestion Worker
21. Case Study: Image Thumbnailer Batch Job
22. Case Study: Webhook Fanout Service
23. Case Study: Metrics Aggregator
24. Migration Recipe: From len(ch) to Proper Sync
25. Testing Refactored Code
26. Benchmarks: Polling vs Event-Driven
27. Anti-Pattern Hunting in Reviews
28. Static Analysis and Lint Rules
29. Self-Assessment
30. Summary

Introduction¶

At the junior level you learned the rule: do not poll len(ch). At the middle level you must justify it, refactor around it, and read it out of legacy code without breaking production. The race is not just "theoretically possible" — it is a particular, reproducible window between two memory operations, and the moment you know exactly which window it is, every variant of the anti-pattern becomes obvious.

This file does four things:

1. Walks the race formally, with step-by-step interleavings and a memory-model diagram, so you can explain it to a colleague without hand-waving.
2. Builds out a catalogue of the correct synchronisation primitives — WaitGroup, done channels, context, errgroup, and close+range — with a working program for each, the failure modes they prevent, and the failure modes they introduce.
3. Walks four production case studies where the pattern hid for months before causing an incident, and shows the diff that fixed each one.
4. Closes with reviewing rules, static checks, benchmarks, and a self-assessment.

After reading you should be able to refactor any polling loop you encounter, justify the change with a memory-model argument, and write tests that prove the new code is free of the race.

Recap of the Anti-Pattern¶

The pattern always looks roughly like this:

// Producer side
jobs := make(chan job, 64)
for _, j := range work {
    go func(j job) { jobs <- j }(j)
}

// "Wait for completion"
for len(jobs) > 0 {
    time.Sleep(5 * time.Millisecond)
}
close(jobs)

It is wrong for five independent reasons. Any one is enough to fail in production.

1. len(jobs) is a snapshot, not a synchronisation point. The value is stale by the time the comparison runs.
2. There is no happens-before edge between the producer's send and the polling goroutine's read of len.
3. The pattern conflates "channel buffer is empty" with "all work is done." A consumer can read the last item and still be processing it.
4. Sleeping is not a substitute for a wake-up signal. Five milliseconds is too long when the system is idle and too short when it is loaded.
5. Closing the channel here happens before the producers finish sending, so producers may panic with send on closed channel.

Every section below tightens one of those points or shows a primitive that makes it irrelevant.

The Formal Race: Step by Step¶

Consider this minimal program. It looks correct to many engineers on first read.

package main

import (
    "fmt"
    "time"
)

func main() {
    ch := make(chan int, 1)

    go func() {
        ch <- 42
    }()

    for len(ch) == 0 {
        time.Sleep(time.Millisecond)
    }

    v := <-ch
    fmt.Println(v)
}

It happens to print 42 on every laptop run. Why is it still a race?

The interleavings¶

Walk the timeline two ways. In each one, "T0", "T1" are wall-clock instants, not scheduler steps.

Interleaving A — Happy path.

Interleaving B — Why this is still a race.

What does len(ch) return at T=4? Go does not guarantee. The runtime does take the channel's internal mutex before reading the buffer count, but the visibility of changes between the send and the read is governed by the Go memory model, and the spec does not promise that "send completed" implies "future reads of len(ch) return the new count."

In practice on amd64 you see the increment; on weak-memory architectures (arm64 under heavy load) you can see the old value because the cache line has not yet been invalidated. This is not a hypothetical: it is reproducible on Graviton instances with GOMAXPROCS=8.

Why time.Sleep cannot rescue you¶

People paper over this race by adding time.Sleep(10 * time.Millisecond) to the polling loop. That changes the probability of seeing the race, not its existence. On loaded production hardware the goroutine that should have sent may still be in the runnable queue 100 milliseconds later. The race exits the laptop and enters production.

Why runtime.Gosched() cannot rescue you¶

The same applies to runtime.Gosched(): it gives the scheduler a hint, but there is no guarantee any particular goroutine will run, nor that any write becomes visible. Gosched is advice, not synchronisation.

Why a Mutex around len cannot rescue you¶

Even if you wrap the call in a mutex, you have only synchronised reads of the mutex. The send into the channel is a different operation — Go does not let you mutex-protect a channel operation. The send's happens-before edge is established by the channel itself, not by your mutex.

The conclusion is that the race is irreducible: no amount of sleeping, yielding, locking, or atomic-ing around the polling loop can fix it, because the fundamental problem is that len(ch) is the wrong observable.

Memory Model Implications¶

The Go memory model, version 2024-04, is explicit:

A send on a channel is synchronized before the completion of the corresponding receive from that channel.

It says nothing about len. There is no clause that links "send happens" to "any future call to len(ch) observes the change." The official rule is that len(ch) is a non-synchronising operation. It reads a counter under the channel's internal lock and returns immediately. That counter's value is observable, but it does not participate in the happens-before relation.

This has three concrete consequences.

Consequence 1: writes before the send are not visible via len¶

If a producer does:

sharedConfig.LoadVersion = 7
ch <- "go"

then a consumer that does <-ch is guaranteed by the memory model to see LoadVersion == 7. A consumer that does if len(ch) > 0 is not. The send is the synchronisation point. len is not.

Consequence 2: len can lag behind reality on weak-memory CPUs¶

On amd64 with its strong memory model, you almost always see the channel counter update promptly. On arm64, ppc64, and riscv64, the channel mutex is still respected (Go uses internal locks), but the moment of visibility is allowed to lag. You may read len(ch) == 0 after a send has structurally completed.

Consequence 3: optimisations are legal¶

The compiler is permitted to hoist a len(ch) read out of a loop because it is not a synchronisation operation. In practice the Go compiler does not do this today — channel ops are opaque to it — but the spec allows it, and tomorrow's compiler may exploit it. Code that relies on len(ch) re-reading the value every iteration is relying on an accident of today's compiler.

Diagram: synchronisation edges that exist and ones that do not¶

producer goroutine                consumer goroutine
-----------------                  ------------------
write x = 7
ch <- x         ───── HB ─────►   y := <-ch     // sees x == 7
                                  (consumer is happy)


producer goroutine                consumer goroutine
-----------------                  ------------------
write x = 7
ch <- x         . . . . . . . .   if len(ch) > 0    // sees what?
                (no HB edge)      // memory model gives no answer

"HB" is happens-before. The first diagram has it; the second does not. That is the entire bug.

Why the Race Detector Sometimes Stays Quiet¶

Engineers report: "We ran with -race in CI for a year and it never caught this." That is not because the code is safe; it is because the race detector instruments shared memory, not channel length. The channel's internal counter is protected by an internal mutex; the detector sees the mutex acquisition as well-formed locking and does not flag the access.

What the detector does catch:

• Two goroutines reading and writing the same plain variable without a lock.
• A goroutine that reads a map while another writes to it (when the runtime catches it directly, not the detector).
• A goroutine that writes to a slice the moment after another goroutine read it without synchronisation.

What the detector does not catch:

• Polling len(ch) while another goroutine sends. Both ops are internally locked.
• Two goroutines that happen to access disjoint indices of the same slice without synchronisation, where the disjointness is determined by external logic.
• Logic races. If your program is wrong but no two goroutines touch the same memory without a lock, the race detector reports nothing.

The lesson: a clean -race run is necessary, not sufficient. Read your synchronisation by hand.

Polling Versus Event-Driven Synchronisation¶

The choice between polling and event-driven coordination is not a stylistic preference; it is the choice between probabilistic correctness and deterministic correctness.

The asymmetry is severe. There is no real trade-off in favour of polling for this use case.

Correct Pattern Catalogue: WaitGroup¶

sync.WaitGroup is the textbook tool for "wait until N goroutines finish." It exposes three operations: Add(delta) to grow the counter, Done() to decrement it, and Wait() to block until it reaches zero.

Minimal working example¶

package main

import (
    "fmt"
    "sync"
)

func main() {
    var wg sync.WaitGroup
    results := make([]int, 10)

    for i := 0; i < 10; i++ {
        wg.Add(1)
        go func(i int) {
            defer wg.Done()
            results[i] = i * i
        }(i)
    }

    wg.Wait()
    fmt.Println(results)
}

There is no len(ch). There is no sleep. The Wait call blocks until the counter is zero, with a happens-before edge between every Done and the return of Wait.

Why this is race-free¶

The Go memory model guarantees:

A call to wg.Done() is synchronized before the return of any call to wg.Wait() that observed the counter reach zero.

That is the explicit happens-before edge that len(ch) lacks. Every write a goroutine performs before calling Done is visible to whoever resumes from Wait.

When to add¶

The deltas matter. Two rules cover almost every mistake.

1. Call Add before spawning the goroutine, not inside it. If you call Add from inside the goroutine, the parent may reach Wait before the child has run and observed Add, returning immediately.

// Wrong
for i := 0; i < 10; i++ {
    go func() {
        wg.Add(1) // racy with Wait
        defer wg.Done()
        work()
    }()
}
wg.Wait() // may return before any goroutine starts

// Right
for i := 0; i < 10; i++ {
    wg.Add(1)
    go func() {
        defer wg.Done()
        work()
    }()
}
wg.Wait()

1. Always pair Add(1) with a matching Done(). Use defer wg.Done() as the first line of the goroutine so a panic does not strand the counter above zero.

When WaitGroup is wrong¶

WaitGroup does not collect results. If the goroutines produce values, you still need a channel for the values themselves. The WaitGroup only tells you "they are all done."

results := make(chan int, len(work))
var wg sync.WaitGroup
for _, w := range work {
    wg.Add(1)
    go func(w job) {
        defer wg.Done()
        results <- process(w)
    }(w)
}
go func() {
    wg.Wait()
    close(results)
}()
for r := range results {
    consume(r)
}

The pattern of "close after wait" inside a separate goroutine is the canonical way to convert N goroutines' worth of output into a single closed channel, which is exactly what range expects.

When WaitGroup is too low-level¶

If you need cancellation or error propagation, jump to errgroup. WaitGroup is unaware of either.

Correct Pattern Catalogue: Done Channel¶

The done channel is the most flexible primitive in Go. It is a chan struct{} whose only operation that matters is closing. When closed, every <-done returns immediately and forever.

Minimal working example¶

package main

import "fmt"

func main() {
    done := make(chan struct{})
    go func() {
        defer close(done)
        fmt.Println("worker doing the thing")
    }()
    <-done
    fmt.Println("worker finished")
}

close(done) is the wake-up signal. <-done is the wait. There is a happens-before edge from the close to the unblocked receive: every write the closer made before closing is visible after the receive returns.

Selecting on done¶

The power of the done channel comes from select. It composes with any other channel.

for {
    select {
    case <-ctx.Done():
        return ctx.Err()
    case item, ok := <-input:
        if !ok {
            return nil
        }
        if err := process(item); err != nil {
            return err
        }
    }
}

The pattern is "wait for the first of these events." Polling cannot do this.

chan struct{} versus chan bool¶

Use chan struct{} to signal "an event happened." struct{} allocates zero bytes, which is a hint to the reader that the channel carries no payload. Use chan bool only when the boolean genuinely conveys information, which is rare.

Closing twice panics¶

The most common mistake with done channels is closing twice. Go panics with close of closed channel. Mitigation:

• One goroutine owns the close. Document it. Test it.
• Use sync.Once.Do(func() { close(done) }) if multiple sites might trigger completion.

var once sync.Once
finish := func() { once.Do(func() { close(done) }) }

Receiving from a closed channel always succeeds¶

This is the property that makes done channels work. Any number of consumers can wait on the same done channel; they all wake when it closes. The channel does not need to be drained — closing is the broadcast.

done := make(chan struct{})
for i := 0; i < 10; i++ {
    go func(i int) {
        <-done
        fmt.Println("worker", i, "exiting")
    }(i)
}
close(done) // wakes all ten

When the done channel is wrong¶

If you need to send a value with the signal — for example, the result of an operation — you may want chan result or chan error. But for pure "done" semantics, prefer chan struct{}.

Correct Pattern Catalogue: Context¶

context.Context is the standard library's bundled done-channel-with-deadline. It exposes:

• Done() <-chan struct{} — the done channel.
• Err() error — what cancelled it (context.Canceled or context.DeadlineExceeded).
• Value(key) any — request-scoped metadata.
• Deadline() (time.Time, bool) — the deadline if one was set.

For wait-for-completion semantics, the relevant pieces are Done() and Err().

Cancellation with context.WithCancel¶

ctx, cancel := context.WithCancel(context.Background())
defer cancel()

go func() {
    if err := serve(ctx); err != nil {
        log.Println("serve error:", err)
    }
}()

shutdownSignal := make(chan os.Signal, 1)
signal.Notify(shutdownSignal, os.Interrupt)
<-shutdownSignal
cancel()

Inside serve, the goroutine watches ctx.Done():

func serve(ctx context.Context) error {
    for {
        select {
        case <-ctx.Done():
            return ctx.Err()
        case req := <-incoming:
            handle(req)
        }
    }
}

Cancellation with context.WithTimeout¶

ctx, cancel := context.WithTimeout(parent, 30*time.Second)
defer cancel()
result, err := callExternal(ctx)

If callExternal honours ctx.Done(), the operation aborts after 30 seconds. The timer fires cancel automatically.

Always call cancel¶

Every WithCancel, WithTimeout, and WithDeadline returns a cancel function. You must call it eventually — defer cancel() is the idiom. Failing to do so leaks the timer and the parent's child slot, which becomes a goroutine leak in long-running processes.

Context is not a WaitGroup¶

ctx.Done() tells the children to stop. It does not wait for them to actually stop. You still need a separate mechanism — usually WaitGroup or errgroup — to wait for the goroutines to exit before the parent returns.

func run(ctx context.Context) {
    var wg sync.WaitGroup
    for i := 0; i < 10; i++ {
        wg.Add(1)
        go worker(ctx, &wg, i)
    }
    wg.Wait() // wait for all workers to exit
}

func worker(ctx context.Context, wg *sync.WaitGroup, id int) {
    defer wg.Done()
    for {
        select {
        case <-ctx.Done():
            return
        case item := <-jobs:
            process(item)
        }
    }
}

The two primitives split the responsibility cleanly: context signals "stop", WaitGroup confirms "stopped."

Correct Pattern Catalogue: errgroup¶

golang.org/x/sync/errgroup combines WaitGroup with context and error propagation. It is the right tool when you have N tasks and you want:

• All of them to run concurrently.
• The first error to cancel the rest.
• A single error return.

Minimal working example¶

package main

import (
    "context"
    "fmt"

    "golang.org/x/sync/errgroup"
)

func main() {
    g, ctx := errgroup.WithContext(context.Background())

    for i := 0; i < 5; i++ {
        i := i
        g.Go(func() error {
            return work(ctx, i)
        })
    }

    if err := g.Wait(); err != nil {
        fmt.Println("at least one failed:", err)
    }
}

func work(ctx context.Context, i int) error {
    select {
    case <-ctx.Done():
        return ctx.Err()
    case <-time.After(10 * time.Millisecond):
        return nil
    }
}

How it composes¶

errgroup.Group.Go calls wg.Add(1), runs the function, and on return calls wg.Done. If the function returned an error and no error has been stored yet, it stores it and cancels the group's context. Every other in-flight goroutine sees ctx.Done() and returns early.

The pattern in one diagram¶

        ┌─ work(ctx, 0) ──── nil
        │
WithCtx ├─ work(ctx, 1) ──── err  ──┐
        │                            │
        ├─ work(ctx, 2) ──── ctx.Err ◄┤   (cancellation cascade)
        │                            │
        └─ work(ctx, 3) ──── ctx.Err ◄┘
                                     │
                                     ▼
                                 g.Wait() returns err

Common mistakes¶

• Forgetting to use the ctx returned by errgroup.WithContext. If your goroutines use the parent context, they will not be cancelled by the first error.
• Capturing loop variables. The i := i shadow is mandatory before Go 1.22; after 1.22 the loop variable is per-iteration but the habit is still safe.
• Mixing g.Go with manual wg.Add — do not. Use one or the other.

When errgroup is wrong¶

• If you want to collect errors from all goroutines, not just the first, you need a manual loop. errgroup discards subsequent errors.
• If you want fan-out where partial failure is acceptable (best-effort broadcast), errgroup is too strict.
• If your tasks return values, you still need a channel or a slice; errgroup does not buffer results.

Correct Pattern Catalogue: Close-and-Range¶

The fifth primitive is not an API but a contract: the sender closes the channel; the receiver ranges over it.

items := make(chan item)

go func() {
    defer close(items)
    for _, x := range source {
        items <- transform(x)
    }
}()

for it := range items {
    consume(it)
}

There is no len. There is no done channel. The range loop exits when close(items) runs and the buffer is drained.

Why this is race-free¶

The memory model says:

The closing of a channel is synchronized before a receive that returns because the channel is closed.

Every write the sender made before close(items) is visible to the receiver after the range loop ends.

When there are multiple senders¶

Many senders, one consumer is harder: who closes? The standard pattern is a separate coordinator goroutine that watches a WaitGroup:

items := make(chan item)
var wg sync.WaitGroup

for _, src := range sources {
    wg.Add(1)
    go func(src source) {
        defer wg.Done()
        for _, x := range src.iterate() {
            items <- transform(x)
        }
    }(src)
}

go func() {
    wg.Wait()
    close(items)
}()

for it := range items {
    consume(it)
}

The WaitGroup tracks producer completion, the close converts that into the range exit, and the consumer never sees a polled length.

Choosing the Right Primitive¶

The decision tree:

Do you need to know "all N goroutines finished"?
├── Yes: do they return errors?
│   ├── Yes: errgroup
│   └── No: WaitGroup
│
└── No: do you need to broadcast "stop now"?
    ├── Yes: does the work have a deadline or cancellation parent?
    │   ├── Yes: context
    │   └── No: closed done channel
    │
    └── No: are you streaming values one direction?
        ├── Yes: close-and-range
        └── No: select on the relevant channels

You should never reach a leaf labelled "poll len(ch)." If the decision tree leads you toward that, the requirement is unclear and you should re-think the design.

Composing Primitives: Real-World Layouts¶

In production code these primitives rarely appear in isolation. The standard layouts:

Layout A: context + WaitGroup¶

func run(ctx context.Context, n int) error {
    var wg sync.WaitGroup
    for i := 0; i < n; i++ {
        wg.Add(1)
        go func(i int) {
            defer wg.Done()
            worker(ctx, i)
        }(i)
    }
    wg.Wait()
    return ctx.Err()
}

Context provides cancellation, WaitGroup confirms shutdown.

Layout B: errgroup + bounded fan-out¶

g, ctx := errgroup.WithContext(ctx)
sem := make(chan struct{}, 8) // limit concurrency to 8
for _, item := range items {
    item := item
    sem <- struct{}{}
    g.Go(func() error {
        defer func() { <-sem }()
        return process(ctx, item)
    })
}
return g.Wait()

errgroup for error propagation, semaphore channel for concurrency limit.

Layout C: pipeline of close-and-range stages¶

nums := source(ctx)
squared := square(ctx, nums)
filtered := filter(ctx, squared)
for v := range filtered {
    fmt.Println(v)
}

Each stage closes its output channel when its input closes. The whole pipeline shuts down by cancelling the context or by the source closing first.

Layout D: producer/consumer with explicit drain¶

items := make(chan job, 64)
done := make(chan struct{})

// producer
go func() {
    defer close(items)
    feed(items)
}()

// consumer
go func() {
    defer close(done)
    for j := range items {
        handle(j)
    }
}()

<-done

The done channel proves both "producer finished" (close happened) and "consumer drained the buffer" (range returned).

WaitGroup Pitfalls You Will Hit¶

sync.WaitGroup is small, but its API has sharp edges.

Pitfall 1: Add inside the goroutine¶

We saw this above. Always Add from the parent before go.

Pitfall 2: re-using a WaitGroup across phases¶

WaitGroup can be re-used, but only after Wait returns and before any new Add raises the counter from zero. Mixing Add and Wait concurrently is a race.

// Wrong: Add concurrent with Wait
var wg sync.WaitGroup
wg.Add(1)
go func() { defer wg.Done(); wg.Add(1); go func() { defer wg.Done(); work() }() }()
wg.Wait() // may race: did the inner Add happen before Wait started?

Use a fresh WaitGroup for each phase, or design so the parent's Add count covers all descendants.

Pitfall 3: panicking inside the goroutine¶

A panic skips ordinary statements but runs deferred functions. So defer wg.Done() is panic-safe, but bare wg.Done() at the end is not.

// Wrong
go func() {
    work()
    wg.Done() // skipped on panic
}()

// Right
go func() {
    defer wg.Done()
    work()
}()

Pitfall 4: passing WaitGroup by value¶

func spawn(wg sync.WaitGroup) { // BAD: copy
    wg.Add(1)
    go func() { defer wg.Done(); work() }()
}

A WaitGroup is a struct with internal counters. Copying it copies the counters, and the parent's Wait cannot see the child's Done. Pass *sync.WaitGroup.

Pitfall 5: Done more than Add¶

Calling wg.Done() when the counter is zero panics with sync: negative WaitGroup counter. This usually happens when an error path returns early after defer wg.Done() was already scheduled in a parent.

Pitfall 6: forgetting to wait¶

for _, w := range work {
    wg.Add(1)
    go process(w, &wg)
}
return // forgot wg.Wait()

The function returns before the goroutines finish. They leak, the WaitGroup is GC'd while they hold a reference, and you see weird behaviour downstream.

Done Channel Pitfalls You Will Hit¶

Pitfall 1: closing a nil channel¶

close(nil) panics. If you have an optional done channel, check it.

Pitfall 2: sending on done¶

Done channels carry no payload. Sending on them works (with a sized buffer), but the receiver cannot distinguish "we sent a value" from "we closed." Always close, never send.

Pitfall 3: leaking goroutines that wait on never-closed done¶

done := make(chan struct{})
go func() {
    <-done
    cleanup()
}()
// forgot to close(done)

The goroutine waits forever. Document who closes. Use defer close(done) at the closer site.

Pitfall 4: race between close and send¶

If multiple goroutines might close, use sync.Once. If multiple goroutines might send on a separate data channel while one goroutine closes the done channel, design so the senders observe <-done and stop sending before close.

for {
    select {
    case <-done:
        return // stop sending
    case work <- next():
    }
}

Pitfall 5: using done as a "kick" rather than a "stop"¶

A done channel is broadcast. Closing it wakes everyone. If you want to wake one specific consumer, that is a different pattern (chan something). Trying to use done as a kick leads to either sending on a chan struct{} (works but counter-idiomatic) or closing and re-making (forbidden — the channel cannot be re-opened).

Context Pitfalls You Will Hit¶

Pitfall 1: ignoring ctx.Done()¶

A goroutine that does not select on ctx.Done() cannot be cancelled. Every long-running loop and every blocking call must honour the context.

// Wrong
for {
    item := <-input
    process(item)
}

// Right
for {
    select {
    case <-ctx.Done():
        return ctx.Err()
    case item := <-input:
        process(item)
    }
}

Pitfall 2: storing context in a struct¶

Idiomatic Go passes ctx as the first parameter of a function. Storing it in a struct ties the struct's lifetime to that particular context, which is almost never what you want. Exception: workers that own their lifecycle (a Server) can store a cancel func and a derived ctx — but pass it down explicitly.

Pitfall 3: context.TODO versus context.Background¶

Background is the root of a context tree. TODO is a placeholder for "I have not yet decided what context belongs here." Use TODO when refactoring or stubbing; replace with the real context as soon as it is known.

Pitfall 4: deriving a context but never cancelling it¶

ctx, _ := context.WithTimeout(parent, 30*time.Second) // BAD: cancel discarded

The timer leaks. Always defer cancel().

Pitfall 5: passing nil context¶

A nil context panics on every method call. If you do not have a context, use context.Background().

Pitfall 6: long deadlines for short work¶

If your operation takes 50 ms but you pass WithTimeout(5 * time.Minute), the operation has effectively no timeout. Right-size the deadline.

Fan-Out Fan-In Without len¶

Fan-out spawns N goroutines that each consume from the same input channel. Fan-in merges their outputs. Done correctly, neither step needs len(ch).

Fan-out¶

func fanOut(ctx context.Context, in <-chan job, workers int) []<-chan result {
    outs := make([]<-chan result, workers)
    for i := 0; i < workers; i++ {
        out := make(chan result)
        outs[i] = out
        go func(out chan<- result) {
            defer close(out)
            for {
                select {
                case <-ctx.Done():
                    return
                case j, ok := <-in:
                    if !ok {
                        return
                    }
                    out <- process(ctx, j)
                }
            }
        }(out)
    }
    return outs
}

Each worker has its own output channel. When the input closes or context cancels, the worker closes its output. Consumers know "this worker is done" by the close, not by len.

Fan-in¶

func fanIn(ctx context.Context, ins ...<-chan result) <-chan result {
    out := make(chan result)
    var wg sync.WaitGroup
    wg.Add(len(ins))
    for _, c := range ins {
        go func(c <-chan result) {
            defer wg.Done()
            for r := range c {
                select {
                case <-ctx.Done():
                    return
                case out <- r:
                }
            }
        }(c)
    }
    go func() {
        wg.Wait()
        close(out)
    }()
    return out
}

wg.Wait() plus close-after-wait makes the merged channel itself rangeable. The consumer iterates with for r := range out and never asks how many items remain.

Worker Pools Without len¶

A worker pool is N long-lived goroutines that pull from a job channel.

func pool(ctx context.Context, jobs <-chan job, n int) error {
    g, ctx := errgroup.WithContext(ctx)
    for i := 0; i < n; i++ {
        g.Go(func() error {
            for {
                select {
                case <-ctx.Done():
                    return ctx.Err()
                case j, ok := <-jobs:
                    if !ok {
                        return nil
                    }
                    if err := handle(ctx, j); err != nil {
                        return err
                    }
                }
            }
        })
    }
    return g.Wait()
}

The pool shuts down when:

• The producer closes jobs. Each worker observes the close and returns nil.
• The context cancels. Each worker observes ctx.Done() and returns ctx.Err().

Either way, g.Wait() is the one-and-only completion signal. Polling has no place here.

Pipelines Without len¶

Pipelines chain stages that transform values:

out1 := stage1(ctx, in)
out2 := stage2(ctx, out1)
out3 := stage3(ctx, out2)
for v := range out3 {
    consume(v)
}

Each stage:

func stageN(ctx context.Context, in <-chan T) <-chan T {
    out := make(chan T)
    go func() {
        defer close(out)
        for {
            select {
            case <-ctx.Done():
                return
            case v, ok := <-in:
                if !ok {
                    return
                }
                select {
                case <-ctx.Done():
                    return
                case out <- transform(v):
                }
            }
        }
    }()
    return out
}

The pipeline cancels by closing the source channel (which cascades) or by cancelling the context (which short-circuits every stage). At no point does any stage call len.

Case Study: Order Ingestion Worker¶

A payment company runs an order ingestion service. Customer orders arrive over HTTP, are validated, enriched with risk scores, and written to a database. The original code:

func (s *Service) Ingest(orders []Order) error {
    jobs := make(chan Order, len(orders))
    for _, o := range orders {
        jobs <- o
    }

    for i := 0; i < 8; i++ {
        go func() {
            for {
                if len(jobs) == 0 {
                    return
                }
                o := <-jobs
                s.process(o)
            }
        }()
    }

    for len(jobs) > 0 {
        time.Sleep(10 * time.Millisecond)
    }
    return nil
}

Three races, two of them silent:

1. The workers' len(jobs) == 0 check races with each other. Two workers can both see len == 1, both attempt <-jobs, and one of them blocks forever.
2. The parent's for len(jobs) > 0 races with the workers' receive. The parent can see len == 0 while a worker is mid-process, return, and miss the result.
3. There is no error handling. s.process returning an error vanishes.

The refactor:

func (s *Service) Ingest(ctx context.Context, orders []Order) error {
    g, ctx := errgroup.WithContext(ctx)
    jobs := make(chan Order)

    g.Go(func() error {
        defer close(jobs)
        for _, o := range orders {
            select {
            case <-ctx.Done():
                return ctx.Err()
            case jobs <- o:
            }
        }
        return nil
    })

    for i := 0; i < 8; i++ {
        g.Go(func() error {
            for o := range jobs {
                if err := s.process(ctx, o); err != nil {
                    return err
                }
            }
            return nil
        })
    }

    return g.Wait()
}

Producer closes jobs after enqueueing. Workers range. errgroup waits and propagates the first error. No len, no sleep, deterministic.

The follow-up incident report measured 0.3% of orders silently dropped under load over a 90-day window. After the refactor: zero drops in the next 18 months.

Case Study: Image Thumbnailer Batch Job¶

A media platform thumbnails a batch of uploaded images every night. The legacy code:

func thumbnail(images []string) {
    work := make(chan string, len(images))
    for _, img := range images {
        work <- img
    }

    for i := 0; i < runtime.NumCPU(); i++ {
        go func() {
            for {
                if len(work) == 0 {
                    return
                }
                img := <-work
                resize(img)
            }
        }()
    }

    for len(work) > 0 {
        time.Sleep(100 * time.Millisecond)
    }
}

Symptom: a small fraction of nightly batches finished with one image unprocessed. The on-call team observed it once, could not reproduce, and added a 1-second sleep at the end. That made it rarer, not impossible.

Root cause: the same race as the ingestion worker. The parent's len == 0 check raced with a worker that had just read the last image and was inside resize. The parent returned; the worker continued; but the parent's return implied "all done," and the next stage proceeded as if every image had been processed.

The refactor:

func thumbnail(ctx context.Context, images []string) error {
    g, ctx := errgroup.WithContext(ctx)
    sem := make(chan struct{}, runtime.NumCPU())
    for _, img := range images {
        img := img
        sem <- struct{}{}
        g.Go(func() error {
            defer func() { <-sem }()
            return resize(ctx, img)
        })
    }
    return g.Wait()
}

g.Wait() blocks until every g.Go call's function has returned. Drift impossible. Concurrency capped by the semaphore.

Case Study: Webhook Fanout Service¶

A webhook fanout service delivers events to N subscribers. The legacy fanout:

type Fanout struct {
    subs []chan Event
}

func (f *Fanout) Publish(e Event) {
    for _, c := range f.subs {
        c <- e
    }
}

func (f *Fanout) Drain() {
    for {
        total := 0
        for _, c := range f.subs {
            total += len(c)
        }
        if total == 0 {
            return
        }
        time.Sleep(10 * time.Millisecond)
    }
}

Drain was used at shutdown to "wait for in-flight events." The bug: a subscriber's buffered channel could have items that the subscriber's goroutine had already read into a local variable but not yet acted on. len showed zero while work was in flight.

The fix moved the wait off len and onto the subscribers themselves:

type Fanout struct {
    subs []*subscriber
}

type subscriber struct {
    events chan Event
    done   chan struct{}
}

func (f *Fanout) Publish(e Event) {
    for _, s := range f.subs {
        s.events <- e
    }
}

func (f *Fanout) Shutdown(ctx context.Context) error {
    for _, s := range f.subs {
        close(s.events)
    }
    for _, s := range f.subs {
        select {
        case <-ctx.Done():
            return ctx.Err()
        case <-s.done:
        }
    }
    return nil
}

Each subscriber's goroutine closes done only after it has drained its events channel and finished handling the last event. The shutdown waits on each done rather than on len(events).

Case Study: Metrics Aggregator¶

A metrics aggregator accumulates counters in shards, flushes them periodically. The legacy flush:

func (a *Aggregator) Flush() Snapshot {
    var s Snapshot
    for i := 0; i < len(a.shards); i++ {
        for len(a.shards[i].in) > 0 {
            // wait for shard to drain
        }
        s.merge(a.shards[i].counts)
    }
    return s
}

The flush busy-waited (no sleep) on each shard's input channel. CPU spiked during flushes; flushes occasionally captured incomplete data because the shard was still copying its counts map while the flush observed len(in) == 0 and proceeded to merge.

The refactor:

type shard struct {
    in     chan event
    flush  chan chan map[string]uint64
}

func (s *shard) run(ctx context.Context) {
    counts := map[string]uint64{}
    for {
        select {
        case <-ctx.Done():
            return
        case e := <-s.in:
            counts[e.key] += e.value
        case reply := <-s.flush:
            snap := maps.Clone(counts)
            reply <- snap
        }
    }
}

func (a *Aggregator) Flush() Snapshot {
    var s Snapshot
    replies := make([]chan map[string]uint64, len(a.shards))
    for i, sh := range a.shards {
        replies[i] = make(chan map[string]uint64, 1)
        sh.flush <- replies[i]
    }
    for _, r := range replies {
        s.merge(<-r)
    }
    return s
}

The flush sends a reply channel to each shard. The shard's main loop services the flush request between event handles, so there is no race between the event accumulation and the snapshot copy.

The cost is one extra channel per flush, paid once per second. The benefit is correct snapshots and a CPU footprint that drops by 12% in production.

Migration Recipe: From len(ch) to Proper Sync¶

A repeatable five-step process when you encounter the anti-pattern in a codebase.

1. Identify the producers and consumers. Who sends to the channel? Who receives? Who owns the close? If the answer to any of these is "unclear," that is the first thing to fix.

2. Pick the primitive. Use the decision tree from earlier. For most worker-pool refactors it is errgroup plus close-and-range. For shutdown flows it is context plus WaitGroup.

3. Refactor the producer side first. Make the producer responsible for closing the channel once it has finished sending. If there are multiple producers, introduce a WaitGroup that closes the channel when all producers Done.

4. Refactor the consumer side. Replace the polling loop with a for x := range ch loop. If consumers need cancellation, switch to select with ctx.Done().

5. Add tests that exercise the race window. Tests in -race mode that send and close from multiple goroutines, plus a "many tiny channels" stress test (go test -race -count=100 -run TestRefactor).

Each step is mechanical. Document the ownership in a code comment so the next person to touch the file does not regress.

Testing Refactored Code¶

Tests that prove the refactor:

Test 1: every input produces an output¶

func TestPoolProcessesAllInputs(t *testing.T) {
    const n = 1000
    in := make([]int, n)
    for i := range in {
        in[i] = i
    }
    got, err := process(context.Background(), in)
    if err != nil {
        t.Fatal(err)
    }
    if len(got) != n {
        t.Fatalf("want %d outputs, got %d", n, len(got))
    }
}

If the old len(ch) polling had dropped inputs, this test would catch it after enough iterations. Combine with go test -count=100 and -race.

Test 2: cancellation aborts in-flight work¶

func TestPoolHonoursContext(t *testing.T) {
    ctx, cancel := context.WithCancel(context.Background())
    in := make([]int, 1_000_000)
    go func() {
        time.Sleep(50 * time.Millisecond)
        cancel()
    }()
    _, err := process(ctx, in)
    if !errors.Is(err, context.Canceled) {
        t.Fatalf("want canceled, got %v", err)
    }
}

A len-polled pool cannot be cancelled mid-flight. The refactored version returns promptly.

Test 3: errors propagate¶

func TestPoolPropagatesErrors(t *testing.T) {
    in := []int{1, 2, 3, -1, 4, 5}
    _, err := process(context.Background(), in)
    if err == nil {
        t.Fatal("expected error for negative input")
    }
}

errgroup.Wait returns the first error. A polling pool with no error channel would have swallowed it.

Test 4: no goroutine leak¶

func TestPoolNoLeak(t *testing.T) {
    before := runtime.NumGoroutine()
    for i := 0; i < 100; i++ {
        _, _ = process(context.Background(), []int{1, 2, 3})
    }
    runtime.GC()
    time.Sleep(50 * time.Millisecond)
    after := runtime.NumGoroutine()
    if after > before+1 {
        t.Fatalf("leaked goroutines: before=%d after=%d", before, after)
    }
}

For a stricter version use go.uber.org/goleak:

func TestMain(m *testing.M) {
    goleak.VerifyTestMain(m)
}

Test 5: stress with -race¶

go test -race -count=200 -timeout=60s ./pool

If the refactor is right, this passes. If it is wrong, run it a few times — concurrency bugs love to hide on the first run.

Benchmarks: Polling vs Event-Driven¶

A representative micro-benchmark.

func BenchmarkPolling(b *testing.B) {
    for i := 0; i < b.N; i++ {
        ch := make(chan int, 100)
        for j := 0; j < 100; j++ {
            go func() { ch <- 1 }()
        }
        for len(ch) > 0 {
            // busy
        }
        // drain
        for j := 0; j < 100; j++ {
            <-ch
        }
    }
}

func BenchmarkWaitGroup(b *testing.B) {
    for i := 0; i < b.N; i++ {
        var wg sync.WaitGroup
        ch := make(chan int, 100)
        for j := 0; j < 100; j++ {
            wg.Add(1)
            go func() { defer wg.Done(); ch <- 1 }()
        }
        wg.Wait()
        close(ch)
        for range ch {
        }
    }
}

func BenchmarkRange(b *testing.B) {
    for i := 0; i < b.N; i++ {
        ch := make(chan int)
        var wg sync.WaitGroup
        wg.Add(100)
        for j := 0; j < 100; j++ {
            go func() { defer wg.Done(); ch <- 1 }()
        }
        go func() { wg.Wait(); close(ch) }()
        for range ch {
        }
    }
}

Indicative numbers on a 2024 M3:

BenchmarkPolling-8        12345 ns/op  busy-wait burns CPU
BenchmarkWaitGroup-8       2340 ns/op  five times faster
BenchmarkRange-8           2510 ns/op  similar to WaitGroup

The "polling" version also pegs a CPU core at 100% during the wait. The other two are dominated by goroutine scheduling overhead, not synchronisation.

This is one input shape; benchmark your own. The qualitative pattern is universal: event-driven wins because it does not spin.

Anti-Pattern Hunting in Reviews¶

Search a repository for the canonical shapes.

grep -nR "for len(" --include="*.go"
grep -nR "if len(.*)" --include="*.go" | grep -v "_test"
grep -nR "cap(" --include="*.go"
grep -nR "time.Sleep" --include="*.go"

Each match is a candidate. Real signals:

• for len(ch) > 0 { time.Sleep(... } — almost certainly the anti-pattern.
• for len(jobs) != 0 { runtime.Gosched() } — disguised polling.
• for { if len(ch) == 0 { break }; ... } — same shape, harder to grep.
• for len(ch) == 0 { time.Sleep(... } — polling for arrival, slightly different but related.

In review comments, refer to the decision tree above and recommend the appropriate primitive. Do not stop at "this is wrong" — show the refactor.

Static Analysis and Lint Rules¶

You can build a go/analysis pass that flags any len(ch) where ch has a channel type, used in a for loop condition.

A simple semgrep rule:

rules:
  - id: poll-len-of-channel
    message: |
      Polling len() on a channel is a race. Use sync.WaitGroup,
      a done channel, or context cancellation.
    patterns:
      - pattern: for len($CH) $OP $N { ... }
      - pattern-not: for len($CH) $OP $N { break }
    languages: [go]
    severity: ERROR

The pattern-not exempts the legitimate "drain the buffer then break" pattern, though even that is rare in good code.

A more nuanced staticcheck rule is harder because len has legitimate uses on channels: for instance, exposing in-flight metrics. Tag those with a comment so the lint stays narrow.

// debug-only: len(ch) reported as a gauge, not used for synchronisation.
metrics.Gauge("queue.depth", float64(len(ch)))

Self-Assessment¶

Answer without re-reading.

1. Define the formal race in for len(ch) > 0 { time.Sleep(...) } in one paragraph. Identify the missing happens-before edge.
2. Why does -race not catch this race?
3. Why is runtime.Gosched() insufficient to fix a polling loop?
4. When should you choose errgroup over WaitGroup?
5. What is the canonical "close after wait" pattern, and which problem does it solve?
6. Show a producer-consumer pair where neither side ever calls len.
7. Why must Add be called by the parent, not by the spawned goroutine?
8. What is the difference between context.Done() and a manually managed done channel?
9. Show a fan-out pattern with a concurrency limit.
10. How would you grep a repository for the anti-pattern in five minutes?

If any answer is shaky, re-read the relevant section.

Summary¶

The wait-for-empty-channel anti-pattern is a logical race, not a memory race. The race detector cannot save you; sleeps cannot save you; locks cannot save you. The only fix is to switch the observable from "channel length" to "channel close" or "WaitGroup zero." The replacement primitives — WaitGroup, done channel, context, errgroup, close-and-range — cover every case. The migration is mechanical. The result is faster, cheaper, and correct.

At the senior level you will see the same patterns one layer deeper: ownership protocols, shutdown sequencing, structured concurrency, drain guarantees, semaphore-based bounding, and the API design that makes the anti-pattern impossible by construction. Go read senior.md.

Appendix A: Reading the Go Memory Model Like a Specification¶

The Go memory model is a short document. Read it once a year. The wait-for-empty-channel anti-pattern is exactly the kind of bug it exists to prevent, and reading the spec end-to-end is the surest way to internalise why len(ch) is non-synchronising.

Section by section¶

Overview. Defines "synchronized before" as the partial order that governs visibility. The key sentence: "If event e1 is synchronized before event e2, then e1 happens before e2." Polling is not in that order.

Memory operations. Distinguishes ordinary reads/writes, synchronizing operations, and atomic operations. len(ch) is an ordinary read. Channel send and receive are synchronizing.

Channel communication. The list of edges:

• A send on a channel is synchronized before the completion of the corresponding receive.
• The closing of a channel is synchronized before a receive that returns because the channel is closed.
• The k'th receive on a channel of capacity C is synchronized before the (k+C)'th send.

len appears in none of these. The third clause is interesting: it gives backpressure a happens-before edge. The first clause is the one that the close-and-range pattern uses; the second one is what the done channel pattern uses.

Locks, Once, atomics. Each defines its own happens-before edge. None of them link a non-channel operation to the channel's internal counter, which is what len(ch) reads.

Incorrect synchronization. The spec gives explicit examples of broken code. Read them. They illustrate the same shape as the wait-for-empty-channel pattern: assuming an operation is synchronizing when it is not.

How to apply it in review¶

When you see a goroutine that depends on observing a value, ask: which happens-before edge guarantees that visibility? If you can name it (send/receive, channel close, mutex unlock, atomic store), the code is fine. If you cannot, the code is wrong, even if it works today.

This is the engineering practice that the wait-for-empty-channel anti-pattern violates. The polling loop has no edge. There is nothing to point to in the spec.

Appendix B: Why Some Languages Have This Pattern And Go Does Not¶

In Java, BlockingQueue.size() is documented as "approximate" but is also a common synchronisation crutch. The Java memory model is more permissive than Go's, but the use of size() for completion detection is just as broken.

In Python, Queue.empty() returns a snapshot. The standard library actively warns against using it for synchronisation: "Because of multithreading/multiprocessing semantics, this is not reliable." Many Python programs use it anyway. The result is the same race.

In Rust, channel types in std::sync::mpsc deliberately omit a len method. The designers concluded that the only correct waits are "receive a value" and "the channel is closed." Programs that need the count must implement it explicitly with an atomic counter, which forces them to think about synchronisation.

Go falls between these. It exposes len(ch) and cap(ch), primarily because the channel is a built-in type and those operators work on every built-in type. The exposure is for diagnostics. The community quickly learned not to use them for synchronisation, and the major lint suites flag the pattern. New code should never feature it.

What Go could do differently¶

Some proposals over the years:

• Remove len(ch) entirely. Rejected — too disruptive, and the operator is useful for observability.
• Issue a vet warning. Rejected — too many false positives (gauge metrics, debug logging).
• Add a third-party staticcheck rule. Accepted by community; SA4031 covers a related but narrower case. No mainline rule for the polling-loop shape.

For now the burden is on engineers to read code carefully. This subsection of the Roadmap is meant to make that easier.

Appendix C: The Bounded Wait Helper¶

A common pattern: you want to wait for completion, but with a maximum duration. The naive shape:

deadline := time.Now().Add(5 * time.Second)
for len(ch) > 0 && time.Now().Before(deadline) {
    time.Sleep(10 * time.Millisecond)
}

Wrong for the usual reasons. The correct shape uses context.WithTimeout and a done channel:

ctx, cancel := context.WithTimeout(parent, 5*time.Second)
defer cancel()
select {
case <-done:
    // success
case <-ctx.Done():
    return ctx.Err()
}

The done channel is the thing being waited on. The context is the deadline. Combined with select they give bounded wait without any polling.

What WaitGroup cannot do here¶

WaitGroup.Wait() does not accept a context. There is no WaitWithTimeout. To get bounded wait you must wrap it:

func waitWithTimeout(wg *sync.WaitGroup, timeout time.Duration) error {
    done := make(chan struct{})
    go func() {
        wg.Wait()
        close(done)
    }()
    select {
    case <-done:
        return nil
    case <-time.After(timeout):
        return errors.New("timeout")
    }
}

This helper is standard. Keep one in your project's internal/syncx package.

What this helper does not do¶

If Wait times out, the goroutines are still running. The helper leaks the inner waiting goroutine until they finish. That is acceptable for shutdown-or-fail use cases but unacceptable for high-frequency calls. The correct architecture is to give the workers a cancellable context, so that on timeout you cancel() and the workers exit promptly.

func waitWithCtx(wg *sync.WaitGroup, ctx context.Context) error {
    done := make(chan struct{})
    go func() {
        wg.Wait()
        close(done)
    }()
    select {
    case <-done:
        return nil
    case <-ctx.Done():
        return ctx.Err()
    }
}