Debounce and Throttle — Senior Level¶

Table of Contents¶

1. Introduction
2. Why "Senior" Means Building the Primitive Yourself
3. Token Bucket — The Mathematics
4. Token Bucket — A Production-Grade Implementation
5. Leaky Bucket — The Mathematics
6. Leaky Bucket — Implementations
7. Token Bucket vs Leaky Bucket — Choosing
8. Sliding Window Counter
9. Sliding Window Log
10. Generic Cell Rate Algorithm (GCRA)
11. The Cost of time.Now
12. Atomic-Based Throttles
13. Lock-Free Counters
14. Sharding for Throughput
15. Debouncer Deep Dive
16. Concurrency-Safe Debouncers
17. Coalescing Debouncers
18. Distributed Throttling — Redis
19. Distributed Throttling — Consensus and Approximation
20. Adaptive Throttling
21. Hierarchical Token Buckets
22. Observability for Rate Limiters
23. Benchmarks and Microbenchmarks
24. Failure Modes and Pathologies
25. Self-Assessment
26. Summary

Introduction¶

At the middle level you learned how to call golang.org/x/time/rate and how to wire a debouncer onto time.AfterFunc. At the senior level the questions change. The library is not a black box; it is a particular implementation of a token bucket with particular trade-offs that you must be able to justify in a design review. The debouncer is not just a one-shot timer; it is a coalescer with a precise contract about which event "wins" and under what timing guarantees.

This document covers the engineering substance behind every production rate limiter you will write: the math of token buckets and leaky buckets, the algorithms used by sliding-window counters and GCRA, the surprisingly expensive cost of reading the clock, the design of lock-free atomic counters, the technique of sharding to escape contention on a single counter, and the patterns for extending a local throttle into a cluster-wide one with Redis or a coordinator. We also revisit debounce with the eye of someone who has been bitten by races inside Timer.Reset and by goroutine leaks when a debouncer outlived its owner.

The patterns here are the ones that ship in real services that handle tens of thousands of requests per second per process, that have to provide rate-limit headers to clients, that have to drop log lines when the disk is full, and that have to debounce a fire-hose of cache invalidations into a single rebuild.

Why "Senior" Means Building the Primitive Yourself¶

A junior reaches for rate.Limiter. A middle uses it correctly with a context and a graceful fallback. A senior knows when rate.Limiter is wrong, when it is right, and how to replace it with a custom structure that fits the workload.

You will reach for a custom implementation in at least four situations:

1. The workload is so hot that the lock inside rate.Limiter becomes a bottleneck. A million calls per second on a single limiter saturates the mutex and your tail latency exposes contention spikes.
2. The semantics are subtly different — for example a true fixed-window counter with reset-on-the-minute boundary, or a leaky bucket with a strict outflow rate that smooths bursts rather than allowing them.
3. The limiter must be distributed across processes. The local token bucket gives wrong totals when ten instances each enforce 100 RPS but you want a global 100 RPS.
4. The limiter must be observable in a way the library does not support — for example exposing the current token count to Prometheus, recording reasons for denial, sampling the wait distribution.

The senior level is built on the understanding that a rate limiter is, at its core, three lines of math and a clock. Once you can write the math you can shape it to fit any workload.

Three properties of every rate limiter¶

Every rate limiter, whether it is a token bucket, a leaky bucket, a sliding window, or a custom contraption, has three properties:

1. Rate — the steady-state number of permitted events per unit of time. Usually expressed as r events per second.
2. Burst — the maximum number of events allowed back-to-back when the limiter has been idle. Usually expressed as b events.
3. Behavior on overflow — what happens when an event would exceed the limit. The choices are block (wait until allowed), drop (reject immediately), or queue (admit but delay).

Any rate limiter can be characterised by these three values. The differences between algorithms come from how they account for time and how they compose under bursty input.

A note on terminology¶

Throughout this document we will use the words "permit" and "token" interchangeably. A "permit" is the abstract unit of admission; a "token" is the concrete representation inside a token bucket. When you "take a token" or "acquire a permit" you are doing the same thing: asking the limiter whether the next event is allowed.

We will also use "request" loosely. A request can be an incoming HTTP call, an outgoing API call, a log line, a metric emission, or any unit of work that you want to rate-limit. The algorithms are the same regardless of what the requests are.

Token Bucket — The Mathematics¶

A token bucket is the dominant rate-limiting algorithm in production systems. It is the model used by golang.org/x/time/rate, by AWS, by GCP, by virtually every cloud rate limiter you will ever encounter.

The model is simple. Imagine a bucket with capacity b. The bucket starts full. Tokens drip in at rate r per second. The bucket cannot hold more than b tokens; surplus tokens spill over and are lost. Each request consumes one token (or n tokens for a weighted request). If there are not enough tokens, the request is denied or blocked.

Why this works¶

The bucket gives you two properties simultaneously:

• Long-term rate: over any sufficiently long interval the average admission rate is at most r, because tokens cannot accumulate beyond b so the total tokens drawn over a long interval is bounded by r * T + b.
• Burst tolerance: an idle period accumulates tokens up to b, so a sudden burst of up to b requests is admitted immediately. This matches the way real workloads arrive: idle for a while, then a flurry.

The mathematical formulation: let t_now be the current time and t_last be the last time the bucket was updated. The number of tokens accumulated since the last update is (t_now - t_last) * r, capped at b. After updating, the bucket has min(b, tokens_old + (t_now - t_last) * r) tokens. A request for k tokens is admitted if and only if tokens >= k, in which case tokens decreases by k.

Continuous vs discrete time¶

Two ways to model the bucket:

1. Continuous: tokens are a real number that grows continuously over time. This is the model used by golang.org/x/time/rate. It is mathematically clean and lock-free updates are easy.
2. Discrete: the bucket holds integer tokens, refilled on a clock tick. This is closer to how a textbook describes the algorithm, but it requires a background goroutine or external tick and the granularity of the tick matters.

Continuous time is almost always the right choice for software rate limiters. There is no reason to have a background goroutine ticking when you can compute the refill on demand.

Worked example¶

Suppose r = 10 tokens per second and b = 20. The bucket starts full with 20 tokens.

At t = 0 ten requests arrive simultaneously. Each consumes one token. The bucket has 10 tokens left.

At t = 1 second we have accumulated 1 * 10 = 10 more tokens, so the bucket would have 10 + 10 = 20 tokens, capped at 20. So the bucket is full again.

At t = 1 thirty requests arrive simultaneously. The first 20 each consume a token. The next 10 are denied (or wait). The bucket is empty.

At t = 1.5 we have accumulated 0.5 * 10 = 5 tokens since the last update. The bucket has 5 tokens. Five of the queued requests can proceed; the rest must keep waiting.

At t = 2 we have accumulated another 0.5 * 10 = 5 tokens. The bucket has 10 tokens. The remaining queued requests proceed.

The total admitted over [0, 2] is 10 + 20 + 5 + 5 = 40, which is less than r * 2 + b = 10 * 2 + 20 = 40. The bound is tight.

The "deficit" formulation¶

An equivalent and often more elegant formulation: instead of counting tokens, count the deficit between the next allowed time and now. Each event advances the deficit by 1 / r. The event is admitted if the deficit is in the past (or within b / r of the future, for burst). This is the GCRA approach, covered later.

Weighted requests¶

Some workloads have requests of varying cost. An expensive endpoint might consume 5 tokens; a cheap one consumes 1. The math still works: replace "one token per request" with "k tokens per request." The bucket is admitted if tokens >= k, otherwise the request waits (k - tokens) / r seconds for enough tokens to accumulate.

This generalises nicely to weighted requests: an LLM endpoint might charge tokens equal to the input length, a database endpoint might charge by the number of rows, a network endpoint might charge by bytes. The bucket model is agnostic to what a "token" represents.

Token Bucket — A Production-Grade Implementation¶

Let's write a token bucket from scratch that is competitive with rate.Limiter for our purposes.

Version 1: a straightforward mutex-protected bucket¶

package tokenbucket

import (
    "sync"
    "time"
)

type Bucket struct {
    mu       sync.Mutex
    rate     float64 // tokens per second
    capacity float64 // maximum tokens
    tokens   float64
    last     time.Time
}

func New(rate, capacity float64) *Bucket {
    return &Bucket{
        rate:     rate,
        capacity: capacity,
        tokens:   capacity,
        last:     time.Now(),
    }
}

func (b *Bucket) Allow() bool {
    return b.AllowN(1)
}

func (b *Bucket) AllowN(n float64) bool {
    b.mu.Lock()
    defer b.mu.Unlock()
    now := time.Now()
    elapsed := now.Sub(b.last).Seconds()
    b.tokens += elapsed * b.rate
    if b.tokens > b.capacity {
        b.tokens = b.capacity
    }
    b.last = now
    if b.tokens >= n {
        b.tokens -= n
        return true
    }
    return false
}

This is simple, correct, and probably fast enough for 100k QPS on a single core. The mutex is the bottleneck under heavier load.

Version 2: pre-computed reciprocal for performance¶

Multiplication is cheaper than division on every CPU we care about. Pre-compute 1/rate once so refill is a multiply, not a divide.

type Bucket struct {
    mu       sync.Mutex
    rate     float64
    capacity float64
    tokens   float64
    last     time.Time
}

func (b *Bucket) refill(now time.Time) {
    elapsed := now.Sub(b.last).Seconds()
    if elapsed <= 0 {
        return
    }
    b.tokens += elapsed * b.rate
    if b.tokens > b.capacity {
        b.tokens = b.capacity
    }
    b.last = now
}

func (b *Bucket) AllowN(n float64) bool {
    b.mu.Lock()
    defer b.mu.Unlock()
    b.refill(time.Now())
    if b.tokens >= n {
        b.tokens -= n
        return true
    }
    return false
}

This factors out the refill logic. The mutex is still held across time.Now(), which on Linux is one VDSO call (~15 ns); the entire AllowN typically runs in 60-80 ns per call when the bucket is hot.

Version 3: wait-aware¶

Sometimes you want the limiter to compute when the next token will be available, so the caller can sleep until then.

func (b *Bucket) Reserve(n float64) time.Duration {
    b.mu.Lock()
    defer b.mu.Unlock()
    now := time.Now()
    b.refill(now)
    if b.tokens >= n {
        b.tokens -= n
        return 0
    }
    deficit := n - b.tokens
    wait := time.Duration(deficit / b.rate * float64(time.Second))
    b.tokens = 0
    b.last = now.Add(wait) // schedule the consumption in the future
    return wait
}

Notice the subtlety: when we reserve a future slot we move b.last forward, ensuring subsequent calls compute the refill from the new "future" anchor. This is exactly what rate.Limiter.Reserve does internally.

If a caller does not honor the wait, the bucket will still be correct on the next call, because the token count was decremented and b.last was advanced. The reservation is "consumed" regardless of what the caller does — there is no API to cancel it. This trade-off keeps the algorithm wait-free internally.

Version 4: cancellable wait¶

func (b *Bucket) Wait(ctx context.Context, n float64) error {
    wait := b.Reserve(n)
    if wait == 0 {
        return nil
    }
    select {
    case <-time.After(wait):
        return nil
    case <-ctx.Done():
        // Best-effort restore: try to return the tokens to the bucket.
        b.mu.Lock()
        b.tokens += n
        if b.tokens > b.capacity {
            b.tokens = b.capacity
        }
        b.mu.Unlock()
        return ctx.Err()
    }
}

The cancellation-aware version is necessary in real services because requests time out, clients disconnect, and a limiter that holds a goroutine for ten seconds when the caller's context expired after two is leaking work. The "restore" branch is a best-effort return of unused tokens — it does not perfectly preserve fairness (a later caller may have consumed in the interim) but it prevents pathological waste.

Version 5: avoiding allocations¶

time.After allocates a time.Timer. For a limiter on a hot path that allocation becomes a measurable cost. Use a time.Timer from a sync.Pool instead.

var timerPool = sync.Pool{
    New: func() interface{} {
        t := time.NewTimer(time.Hour)
        if !t.Stop() {
            <-t.C
        }
        return t
    },
}

func getTimer(d time.Duration) *time.Timer {
    t := timerPool.Get().(*time.Timer)
    t.Reset(d)
    return t
}

func putTimer(t *time.Timer) {
    if !t.Stop() {
        select {
        case <-t.C:
        default:
        }
    }
    timerPool.Put(t)
}

func (b *Bucket) Wait(ctx context.Context, n float64) error {
    wait := b.Reserve(n)
    if wait == 0 {
        return nil
    }
    t := getTimer(wait)
    defer putTimer(t)
    select {
    case <-t.C:
        return nil
    case <-ctx.Done():
        b.mu.Lock()
        b.tokens += n
        if b.tokens > b.capacity {
            b.tokens = b.capacity
        }
        b.mu.Unlock()
        return ctx.Err()
    }
}

This pool reduces allocations from roughly one timer per Wait call (the channel inside time.After) to near zero in steady state. On a workload of 100k Wait calls per second the difference is measurable: an extra megabyte of garbage per second versus negligible allocation.

Version 6: tight-path AllowN with no allocations¶

The bare AllowN already has no heap allocations, but time.Now is a non-trivial call. Profiling on a hot service typically shows time.Now accounting for 5-15% of CPU when the limiter is the bottleneck. We will discuss techniques for amortising time.Now later in this document.

Picking initial state¶

A subtle question: when the bucket is first constructed, should it start full or empty?

Starting full means the first burst of b events is immediately admitted. This is friendly to clients that send a small burst at startup (cache warmup, batch initial sync) but it allows a "thundering herd" effect when many processes start simultaneously.

Starting empty is conservative: every request from time zero is subject to the rate. This produces smoother startup behaviour but can frustrate clients that legitimately need a startup burst.

The conventional default — used by rate.Limiter — is to start full. Override it if your workload prefers conservative startup.

Choosing r and b¶

The two parameters interact subtly:

• r is the long-term rate. Pick it to match the downstream system's capacity.
• b is the burst. Pick it to match the latency budget you can tolerate for a backlog to clear.

If the bucket runs full and a burst of b requests arrives, they all clear immediately. The next burst must wait until tokens refill. So a large b gives bursty workloads good latency but a small b smooths the load. The right answer depends on the downstream tolerance.

A common heuristic: b = r * 1 (i.e. one second of refill) gives a one-second burst tolerance and smoothes longer bursts. For UI workloads b = r * 0.5 is common; for batch workloads b = r * 5 or more is reasonable.

Leaky Bucket — The Mathematics¶

The leaky bucket is the older sibling of the token bucket and is sometimes confused with it. The two algorithms have different operational semantics.

The model: imagine a bucket with capacity c and a hole at the bottom that drains at rate r per second. Each request adds one unit of water to the bucket. If the bucket would overflow (the water level exceeds c), the request is denied.

The crucial difference from a token bucket: the leaky bucket smooths the outflow. The token bucket smooths the inflow but allows bursts at the output. The leaky bucket guarantees the output rate is at most r, never more. There are no bursts at the output.

Two leaky bucket variants¶

There are actually two variants of the leaky bucket, frequently conflated:

1. Leaky bucket as a meter (sometimes called "leaky bucket counter"). The bucket level is tracked but the requests are not delayed — they are admitted or denied based on the level. This is operationally similar to a token bucket and is sometimes used interchangeably.

2. Leaky bucket as a queue (sometimes called "leaky bucket scheduler"). The bucket is a FIFO queue with capacity c. Requests are enqueued and dequeued at rate r. This adds latency but absolutely smooths the output rate.

The "queue" variant is what gives the leaky bucket its distinctive smoothing behaviour. The "meter" variant is essentially a renamed token bucket with capacity flipped.

Worked example — meter variant¶

Suppose r = 10 per second and c = 20. The bucket starts empty.

At t = 0 ten requests arrive. The bucket fills to 10. All are admitted.

At t = 0.5 ten more requests arrive. Since the last update, the bucket has drained by 0.5 * 10 = 5, leaving 5. The ten new requests would push it to 15, still under 20. All admitted.

At t = 1 thirty more requests arrive. The bucket has drained by another 0.5 * 10 = 5, leaving 10. The next 10 requests fit, pushing it to 20. The remaining 20 do not fit and are denied.

The cumulative admitted is 10 + 10 + 10 = 30 in 1 second, which is r * 1 + c = 10 + 20 = 30. The bound matches the token bucket bound exactly.

Worked example — queue variant¶

Same parameters. At t = 0 ten requests arrive. They join the queue. The queue contains 10 items.

At t = 1 the queue has been draining at 10 per second, so all ten have been dispatched at evenly spaced times: t = 0, t = 0.1, t = 0.2, ..., t = 0.9. The queue is empty at t = 1.

At t = 1 thirty more requests arrive. They join the queue, but the queue has capacity 20, so the last 10 are denied. The remaining 20 are dispatched at t = 1, 1.1, 1.2, ..., 2.9.

The output rate is exactly 10 per second. The maximum delay is 2 seconds (the time to clear a full queue of 20 at rate 10).

This smoothing comes at a cost: latency. The queue variant has bounded latency c / r for any admitted request.

When to choose leaky bucket over token bucket¶

Choose leaky bucket (queue variant) when:

• The downstream system cannot handle bursts at all. A serial-port driver, a single-threaded legacy service, a cache that thrashes when many writes arrive at once.
• The smoothed delivery has business value. Pacing outgoing email so the recipient SMTP server does not greylist you. Spacing out scraping requests so you do not look like a bot.
• You want a hard upper bound on latency. The queue-variant leaky bucket gives latency <= c / r.

Choose token bucket when:

• The downstream system can handle bursts.
• You care about low latency for the common case (most requests, the bucket has tokens, they are immediate).
• You want simple math and easy distributed coordination.

Leaky Bucket — Implementations¶

Meter variant¶

package leakybucket

import (
    "sync"
    "time"
)

type MeterBucket struct {
    mu       sync.Mutex
    rate     float64 // drain rate in units per second
    capacity float64
    level    float64
    last     time.Time
}

func NewMeter(rate, capacity float64) *MeterBucket {
    return &MeterBucket{
        rate:     rate,
        capacity: capacity,
        last:     time.Now(),
    }
}

func (b *MeterBucket) Allow() bool {
    return b.AllowN(1)
}

func (b *MeterBucket) AllowN(n float64) bool {
    b.mu.Lock()
    defer b.mu.Unlock()
    now := time.Now()
    elapsed := now.Sub(b.last).Seconds()
    b.level -= elapsed * b.rate
    if b.level < 0 {
        b.level = 0
    }
    b.last = now
    if b.level+n <= b.capacity {
        b.level += n
        return true
    }
    return false
}

This is essentially the token bucket flipped: instead of decrementing tokens, we increment the level, and the refill drains the level. The math is symmetric.

Queue variant¶

package leakybucket

import (
    "context"
    "errors"
    "sync"
    "time"
)

type QueueBucket struct {
    mu        sync.Mutex
    rate      time.Duration // 1/r seconds between drains
    capacity  int
    pending   int
    nextDrain time.Time
    cond      *sync.Cond
}

func NewQueue(ratePerSec, capacity int) *QueueBucket {
    b := &QueueBucket{
        rate:     time.Second / time.Duration(ratePerSec),
        capacity: capacity,
    }
    b.cond = sync.NewCond(&b.mu)
    return b
}

var ErrFull = errors.New("queue full")

func (b *QueueBucket) Submit(ctx context.Context) error {
    b.mu.Lock()
    for {
        now := time.Now()
        if b.nextDrain.Before(now) {
            b.nextDrain = now
        }
        // Drain virtual slots that have elapsed.
        for b.pending > 0 && !b.nextDrain.After(now) {
            b.pending--
            b.nextDrain = b.nextDrain.Add(b.rate)
        }
        if b.pending < b.capacity {
            b.pending++
            mySlot := b.nextDrain.Add(time.Duration(b.pending-1) * b.rate)
            b.mu.Unlock()
            wait := time.Until(mySlot)
            if wait <= 0 {
                return nil
            }
            select {
            case <-time.After(wait):
                return nil
            case <-ctx.Done():
                return ctx.Err()
            }
        }
        // Queue full: wait for a drain.
        next := b.nextDrain
        b.mu.Unlock()
        wait := time.Until(next)
        if wait < 0 {
            wait = 0
        }
        select {
        case <-time.After(wait):
        case <-ctx.Done():
            return ctx.Err()
        }
        b.mu.Lock()
    }
}

This is more complex than the meter variant because we track virtual slot times for each pending item. The implementation is illustrative but not optimal — a real implementation would use a heap or a delay queue.

A simpler and more common approach: implement the queue variant by combining a buffered channel with a goroutine that drains it on a time.Ticker. The channel's buffer size is the bucket capacity, and the ticker enforces the drain rate.

type SimpleQueueBucket struct {
    in   chan struct{}
    out  chan struct{}
    rate time.Duration
    stop chan struct{}
}

func NewSimpleQueue(ratePerSec, capacity int) *SimpleQueueBucket {
    b := &SimpleQueueBucket{
        in:   make(chan struct{}, capacity),
        out:  make(chan struct{}),
        rate: time.Second / time.Duration(ratePerSec),
        stop: make(chan struct{}),
    }
    go b.drain()
    return b
}

func (b *SimpleQueueBucket) drain() {
    t := time.NewTicker(b.rate)
    defer t.Stop()
    for {
        select {
        case <-b.stop:
            return
        case <-t.C:
            select {
            case <-b.in:
                b.out <- struct{}{}
            default:
            }
        }
    }
}

func (b *SimpleQueueBucket) Submit(ctx context.Context) error {
    select {
    case b.in <- struct{}{}:
    case <-ctx.Done():
        return ctx.Err()
    default:
        return ErrFull
    }
    select {
    case <-b.out:
        return nil
    case <-ctx.Done():
        return ctx.Err()
    }
}

func (b *SimpleQueueBucket) Stop() {
    close(b.stop)
}

This is concise, correct, and lets the runtime do the scheduling work. The downside: it requires a long-lived goroutine and the ticker keeps the scheduler active even when no requests arrive.

Token Bucket vs Leaky Bucket — Choosing¶

Let's make the choice concrete with three scenarios.

Scenario 1: rate-limiting an external API client¶

You make calls to a payment provider that allows 100 RPS with a burst of 200. You want to maximise throughput while respecting the limit.

Choose token bucket. The provider explicitly accommodates bursts. A leaky bucket would needlessly delay requests during quiet periods. Set r = 100, b = 200. Use rate.Limiter.

Scenario 2: rate-limiting outgoing email¶

You send marketing email through an SMTP relay. The relay limits you to 60 emails per minute. If you exceed it your IP is throttled or blocked.

Choose leaky bucket (queue variant). The relay cares about the smoothed rate. A token bucket would let 60 emails through in the first second, then nothing for a minute, then 60 more — which looks like spam behaviour and may get you flagged. The queue variant smooths the output to one email per second.

Scenario 3: rate-limiting a background job worker¶

Your worker pulls from a queue and processes jobs. You want to cap the worker at 1000 jobs per minute to avoid swamping the database. Jobs arrive in bursts.

Choose token bucket. The database can handle bursts (it has its own buffering). Smooth delivery is a non-goal. Set r = 1000/60, b = 1000. The worker can process up to 1000 jobs as fast as it likes when starting, then settles into the long-term rate.

Scenario 4: protecting a public API endpoint¶

You expose an endpoint that authenticated users can call. You want each user limited to 100 requests per minute, but you do not want to penalise users for legitimate bursts.

Choose token bucket with per-user keys. Set r = 100/60 ≈ 1.67, b = 100. Users can burst up to 100 requests if they have been quiet, but cannot sustain more than 100 per minute.

Hybrid: leaky-meter feeding a token bucket¶

A common architecture for protecting a downstream from a public-facing service:

[clients] -> [public token bucket per IP] -> [internal leaky meter to backend] -> [backend]

The token bucket gives clients a friendly burst-aware experience. The leaky bucket gives the backend a hard rate limit. The two compose naturally and give you defence in depth.

Sliding Window Counter¶

The sliding window counter is the third big algorithm in this space. It avoids the "bucket reset" problem of fixed-window counters while being cheaper than a sliding window log.

Fixed window — and its problem¶

A fixed window counter divides time into windows (say, one minute each) and counts requests per window. If the count exceeds the limit, the request is denied. At the boundary, the count resets.

The problem: a client can get 2x the intended rate by clustering requests around the boundary. If the limit is 100 per minute, the client sends 100 in the last second of minute 1 and 100 in the first second of minute 2 — that is 200 requests in two seconds.

Sliding window log — the gold standard¶

Keep a log of every request's timestamp. To check a request, count how many timestamps are within the last T seconds. If under the limit, admit; record the timestamp.

This is exact but expensive: memory proportional to the number of requests in the window, and the count is O(log N) (binary search the sorted log) or O(N) (linear scan). For a single host at 10k RPS with a 60-second window that is 600k entries, which is workable but not free.

Sliding window counter — the practical approximation¶

Instead of a full log, store the count for the current and previous fixed windows. Interpolate.

Let c_current be the count in the current window, c_prev the count in the previous window, and f the fraction of the current window elapsed (between 0 and 1). The estimated count in the trailing window of one window-width is:

estimate = c_prev * (1 - f) + c_current

Intuition: (1 - f) is the fraction of the previous window that still lies within the trailing window. The estimate weights the previous window's count by how much of it still applies.

If estimate < limit, admit and increment c_current. Otherwise deny.

package sliding

import (
    "sync"
    "time"
)

type Counter struct {
    mu         sync.Mutex
    windowSize time.Duration
    limit      int
    current    int
    previous   int
    windowEnd  time.Time
}

func NewCounter(windowSize time.Duration, limit int) *Counter {
    return &Counter{
        windowSize: windowSize,
        limit:      limit,
        windowEnd:  time.Now().Add(windowSize),
    }
}

func (c *Counter) Allow() bool {
    c.mu.Lock()
    defer c.mu.Unlock()
    now := time.Now()
    for now.After(c.windowEnd) {
        c.previous = c.current
        c.current = 0
        c.windowEnd = c.windowEnd.Add(c.windowSize)
    }
    elapsed := c.windowSize - c.windowEnd.Sub(now)
    f := float64(elapsed) / float64(c.windowSize)
    estimate := float64(c.previous)*(1-f) + float64(c.current)
    if int(estimate) < c.limit {
        c.current++
        return true
    }
    return false
}

The error of the estimate is bounded by c_prev / 2 in the worst case, because the assumption that requests in the previous window were uniformly distributed is approximate. For most workloads the error is small (a few percent), and the savings in memory and CPU are enormous compared to the full log.

When sliding-window counter is the right tool¶

• Per-IP rate limiting where the rate is moderate (tens to thousands per minute) and you cannot store every request timestamp.
• Distributed rate limiting where the storage cost of a log per key is prohibitive.
• Cases where the bucket-based limiters' "burst at the start" is undesirable.

Sliding window counter vs token bucket¶

Both algorithms enforce a long-term rate. The key behavioural differences:

1. Token bucket allows bursts up to b; sliding window strictly enforces the rate over the trailing window.
2. Token bucket is O(1) per call; sliding window is O(1) per call with the counter approximation.
3. Token bucket is friendlier to legitimate clients; sliding window is harder for attackers to game.

For per-user limits on a public API, sliding window counter is a common choice because it is harder to game than fixed window and cheaper than a log.

Sliding Window Log¶

When you need exact counts in a sliding window (no approximation, no boundary effects), the log is the only option.

package slidinglog

import (
    "sync"
    "time"
)

type Log struct {
    mu         sync.Mutex
    windowSize time.Duration
    limit      int
    entries    []time.Time
}

func New(windowSize time.Duration, limit int) *Log {
    return &Log{
        windowSize: windowSize,
        limit:      limit,
        entries:    make([]time.Time, 0, limit),
    }
}

func (l *Log) Allow() bool {
    l.mu.Lock()
    defer l.mu.Unlock()
    now := time.Now()
    cutoff := now.Add(-l.windowSize)
    // Drop entries older than the window.
    i := 0
    for i < len(l.entries) && l.entries[i].Before(cutoff) {
        i++
    }
    l.entries = l.entries[i:]
    if len(l.entries) < l.limit {
        l.entries = append(l.entries, now)
        return true
    }
    return false
}

Performance characteristics:

• O(K) per call where K is the number of expired entries. Amortised this is O(1) per call because each entry is removed exactly once.
• Memory proportional to the number of in-window entries. With a limit N the memory is O(N).

For most rate-limiting use cases the log is overkill, but for billing-grade or compliance-grade limits — where every request must be exactly accounted for — the log is the right tool.

Trimming the log¶

The naive trim above moves the head pointer, but the underlying slice keeps growing without shrinking. After a burst the slice may hold the maximum capacity even when only a few entries are live. Two fixes:

1. Use a ring buffer of fixed capacity equal to the limit. New entries overwrite the oldest. This bounds memory at exactly limit entries.
2. Periodically copy live entries to a new slice. Trades CPU for memory.

type RingLog struct {
    mu         sync.Mutex
    windowSize time.Duration
    entries    []time.Time
    head       int
    size       int
}

func NewRing(windowSize time.Duration, limit int) *RingLog {
    return &RingLog{
        windowSize: windowSize,
        entries:    make([]time.Time, limit),
    }
}

func (l *RingLog) Allow() bool {
    l.mu.Lock()
    defer l.mu.Unlock()
    now := time.Now()
    cutoff := now.Add(-l.windowSize)
    // Trim from the head.
    for l.size > 0 && l.entries[l.head].Before(cutoff) {
        l.head = (l.head + 1) % len(l.entries)
        l.size--
    }
    if l.size < len(l.entries) {
        l.entries[(l.head+l.size)%len(l.entries)] = now
        l.size++
        return true
    }
    return false
}

The ring buffer is what you use when memory matters and the limit is fixed at construction.

Generic Cell Rate Algorithm (GCRA)¶

GCRA is the rate-limiting algorithm used in ATM (asynchronous transfer mode) networks and increasingly in software because of its excellent properties: it is O(1), allocation-free, and uses a single floating-point or integer value per limiter.

The idea¶

Instead of tracking tokens or counts, track a single value: the "theoretical arrival time" (TAT). This is the time at which the next cell (request) is "scheduled" to arrive. If a request arrives before TAT minus the tolerance, deny. Otherwise advance TAT by the period.

Two parameters: - T = 1/rate: the period between requests at steady state. - tau: the tolerance, equivalent to (b - 1) * T where b is the burst.

The algorithm:

On request at time `now`:
    if now < TAT - tau: deny
    else: TAT = max(now, TAT) + T; admit

Why this works¶

The invariant: TAT - tau <= now <= TAT at admission time. This means the request is within tau of "schedule" (early by at most tau, never later). Over a long interval the average period is T, so the long-term rate is 1/T. Over a short interval the request can be up to tau / T early, giving burst tolerance.

Implementation¶

package gcra

import (
    "sync"
    "time"
)

type GCRA struct {
    mu        sync.Mutex
    period    time.Duration // T = 1/rate
    tolerance time.Duration // tau
    tat       time.Time
}

func New(ratePerSec, burst float64) *GCRA {
    period := time.Duration(float64(time.Second) / ratePerSec)
    tolerance := time.Duration(float64(burst-1) * float64(period))
    return &GCRA{period: period, tolerance: tolerance, tat: time.Now()}
}

func (g *GCRA) Allow() bool {
    g.mu.Lock()
    defer g.mu.Unlock()
    now := time.Now()
    if now.Before(g.tat.Add(-g.tolerance)) {
        return false
    }
    newTAT := g.tat
    if now.After(newTAT) {
        newTAT = now
    }
    newTAT = newTAT.Add(g.period)
    g.tat = newTAT
    return true
}

func (g *GCRA) AllowAt(now time.Time) (bool, time.Duration) {
    g.mu.Lock()
    defer g.mu.Unlock()
    if now.Before(g.tat.Add(-g.tolerance)) {
        retry := g.tat.Add(-g.tolerance).Sub(now)
        return false, retry
    }
    newTAT := g.tat
    if now.After(newTAT) {
        newTAT = now
    }
    newTAT = newTAT.Add(g.period)
    g.tat = newTAT
    return true, 0
}

AllowAt returns the time the caller would need to wait before retrying. This is a clean API for clients that want to back off and retry rather than block.

GCRA's advantages¶

• Single state value: tat is the only mutable field. This is friendly to atomic implementations.
• O(1) time: no loops, no allocations.
• Burst semantics match token bucket: GCRA with tau = (b-1) * T is mathematically equivalent to a token bucket with rate 1/T and burst b.

Atomic GCRA¶

Because GCRA's state is one value, we can implement it lock-free with a CAS loop:

package gcra

import (
    "sync/atomic"
    "time"
)

type AtomicGCRA struct {
    period    int64 // nanoseconds
    tolerance int64
    tat       atomic.Int64
}

func NewAtomic(ratePerSec, burst float64) *AtomicGCRA {
    period := int64(float64(time.Second) / ratePerSec)
    tolerance := int64(float64(burst-1) * float64(period))
    g := &AtomicGCRA{period: period, tolerance: tolerance}
    g.tat.Store(time.Now().UnixNano())
    return g
}

func (g *AtomicGCRA) Allow() bool {
    nowNs := time.Now().UnixNano()
    for {
        old := g.tat.Load()
        if nowNs < old-g.tolerance {
            return false
        }
        newTAT := old
        if nowNs > newTAT {
            newTAT = nowNs
        }
        newTAT += g.period
        if g.tat.CompareAndSwap(old, newTAT) {
            return true
        }
    }
}

Under low contention this is dramatically faster than the mutex version (the CAS retries are rare). Under high contention the CAS loop can spin, but in practice for rate limiters the contention is typically low because most calls are rejected quickly (failures do not retry CAS).

We will return to atomic implementations in detail when we discuss lock-free counters.

The Cost of time.Now¶

Every rate limiter calls time.Now at least once per admission decision. On a hot path this matters.

What time.Now actually costs¶

On Linux x86_64, Go's time.Now uses the kernel's VDSO (virtual dynamic shared object), which avoids a syscall and reads the time directly from a memory-mapped page. The CPU instruction is rdtsc or rdtscp followed by some arithmetic. Typical cost:

• Linux x86_64: ~15-25 ns
• Linux arm64: ~20-30 ns
• macOS: ~30-50 ns (no VDSO; uses commpage but slightly slower)
• Windows: ~25-40 ns

On a CPU that runs at 3 GHz with an IPC of 3, 25 ns is roughly 225 useful instructions. That is more than the entire body of a hot Allow() call. So time.Now can easily be the dominant cost of a rate limiter.

Why is it not faster?¶

time.Now does more than rdtsc:

1. It reads the current clock source state (kernel-shared page on Linux).
2. It converts from raw cycles to nanoseconds using a scaling factor that accounts for clock drift.
3. It optionally reads the monotonic and wall clocks separately.
4. It wraps the result in a time.Time struct (24 bytes on 64-bit Go).

Go added the "monotonic clock embedded in time.Time" feature in Go 1.9, which doubles the size of the returned value but enables correct duration subtraction even when the wall clock jumps.

Caching time.Now¶

If you have many calls per second that can tolerate millisecond-stale time values, you can amortise time.Now by reading it once per tick and caching.

package fasttime

import (
    "sync/atomic"
    "time"
)

var nowNs atomic.Int64

func init() {
    nowNs.Store(time.Now().UnixNano())
    go func() {
        t := time.NewTicker(time.Millisecond)
        for range t.C {
            nowNs.Store(time.Now().UnixNano())
        }
    }()
}

func Now() int64 {
    return nowNs.Load()
}

fasttime.Now() returns the cached value, accurate to within ~1 ms. The cost of Now is a single atomic load (~1 ns). This is 15-25x faster than time.Now.

The trade-offs:

• 1 ms staleness. For a 100 RPS limiter the timing error is negligible. For a 100k RPS limiter the error is 100 events at the boundary — still well under the burst.
• A background ticker. The ticker itself is cheap but it does prevent the runtime from being entirely idle.
• A global. The cached time is process-wide; you cannot have multiple cached clocks.

For most rate limiters operating below 10k RPS, time.Now directly is fine. For higher rates fasttime is a measurable win.

Per-CPU time.Now caching¶

A more advanced technique: cache time.Now per logical CPU using runtime_procPin (a private runtime function exposed in golang.org/x/sys/cpu indirectly via runtime.LockOSThread or sharding tricks). Each CPU reads its own cache, which avoids contention on the global atomic.

This is rarely necessary in practice. By the time you need it you should be considering whether your design has too much contention on a single counter.

time.Now and clock skew¶

Go's time.Now uses the monotonic clock for duration calculations, immune to wall-clock jumps from NTP. Rate limiters always want monotonic time. If you compare two time.Time values with Sub, you are using monotonic time. If you call UnixNano() you get wall-clock nanoseconds, which can jump backward — be careful when caching UnixNano() and subtracting.

For our atomic GCRA above, we used UnixNano() for portability. In production, prefer time.Time.Sub with the monotonic reading, or use runtime.nanotime (unexported but accessible via //go:linkname) for the ultimate hot path.

The "now hint" trick¶

Some APIs let the caller pass in now:

func (g *GCRA) AllowAt(now time.Time) bool { ... }

This lets a caller that already has now (e.g., from a request log or a tracing span) avoid calling time.Now again. In a hot HTTP handler that records the request time anyway, the savings are real: one time.Now per request instead of two or three.

Atomic-Based Throttles¶

For very hot paths the mutex inside a token bucket can be the dominant cost. The first optimisation is to replace it with atomic operations.

Why atomic is faster than mutex¶

A sync.Mutex in Go is essentially a futex-backed atomic with fallback to scheduler queueing under contention. The uncontended fast path is ~15-25 ns. The contended path involves goroutine parking, OS thread interaction, and is much slower — easily a microsecond or more.

A sync/atomic.CompareAndSwap on Intel x86_64 is a single LOCK CMPXCHG instruction. Uncontended cost: ~5-10 ns. Contended cost: still microseconds-range because the cache line bounces between cores, but no scheduler involvement.

For a workload of mostly-uncontended limiter calls (say, many cores each calling once per request), atomic is consistently 2-3x faster than mutex. Under high contention the gap narrows because cache-line bouncing dominates.

Atomic GCRA recap¶

We saw above an atomic GCRA. Let's repeat it with more commentary:

type AtomicGCRA struct {
    period    int64
    tolerance int64
    tat       atomic.Int64 // 8 bytes, aligned to a cache line if isolated
}

func (g *AtomicGCRA) Allow() bool {
    nowNs := time.Now().UnixNano()
    for {
        old := g.tat.Load()
        if nowNs < old-g.tolerance {
            return false
        }
        newTAT := old
        if nowNs > newTAT {
            newTAT = nowNs
        }
        newTAT += g.period
        if g.tat.CompareAndSwap(old, newTAT) {
            return true
        }
    }
}

Notice that the deny path returns without a CAS. This means denied requests are free — they read the tat, decide no, and return. Only admitted requests modify state. For a heavily-throttled service where most requests are denied, this is a big win.

Atomic token bucket — is it possible?¶

Yes, but harder than GCRA because the token bucket has two coupled state variables: the token count and the last refill time. We can pack them into a single uint64 if we are clever.

package atomicbucket

import (
    "sync/atomic"
    "time"
)

// state encodes [tokens:32 bits | lastNs:32 bits]
// tokens is fixed-point with 16 bits of fractional part: real = tokens / 65536
// lastNs is the last update time as ns since process start, mod 2^32 (~4.3 sec window)
// This is too narrow for general use; we'll discuss a wider encoding next.

Packing two values into a 64-bit atomic only works if both values fit. For a token bucket with reasonable token counts (millions) and a long-running process, 64 bits is not enough for both.

The practical approach: use a uint64 for the bucket state and accept that the "last update" time is relative to a reference point that resets periodically. Or use a sync.Mutex for the bucket — the contention is usually fine for the workloads where token buckets are appropriate.

A simpler alternative: use atomic for the cheap "Allow" path and fall back to mutex for the slow "wait" path.

type HybridBucket struct {
    mu       sync.Mutex
    rate     float64
    capacity float64
    state    atomic.Pointer[bucketState]
}

type bucketState struct {
    tokens float64
    last   int64 // unixnano
}

func (b *HybridBucket) Allow() bool {
    nowNs := time.Now().UnixNano()
    for {
        old := b.state.Load()
        elapsed := float64(nowNs-old.last) / 1e9
        tokens := old.tokens + elapsed*b.rate
        if tokens > b.capacity {
            tokens = b.capacity
        }
        if tokens < 1 {
            return false
        }
        ns := &bucketState{tokens: tokens - 1, last: nowNs}
        if b.state.CompareAndSwap(old, ns) {
            return true
        }
    }
}

Each successful admission allocates a new bucketState. That allocation is the cost of avoiding the mutex. For a workload with ~50% denial rate this is a wash; for a mostly-admitted workload the allocations dominate and mutex wins.

A variant uses sync.Pool for the state objects, but then you need careful tracking of which states are still referenced — and you can no longer trivially deduce "the new state is mine because CAS succeeded."

The CAS retry storm¶

Under high contention an atomic CAS loop can degenerate into a retry storm: every goroutine reads the state, computes a new state, fails the CAS, and retries. Cache-line bouncing slows everyone down. The total throughput can be lower than with a mutex because the mutex serialises and the CAS does not.

Symptoms: - CPU utilisation high, throughput low. - Profiles show most time in the atomic package or in the CAS loop. - Adding more goroutines makes things slower, not faster.

Solutions: - Add jitter or backoff between retries. - Use sharding (covered next). - Fall back to a mutex; the workload may not actually benefit from atomic.

Lock-Free Counters¶

A counter that records the number of admissions, denials, or any other metric needs to be incremented without locking. Pre-Go 1.19 this was done with atomic.AddInt64. Since Go 1.19 there is atomic.Int64.

Single-counter contention¶

A naive counter:

var counter atomic.Int64

func record() {
    counter.Add(1)
}

Looks fine. On a single core: ~5 ns per increment. On 16 cores all incrementing: ~150-300 ns per increment due to cache line contention. The line that holds the counter bounces between cores' L1 caches.

This is the classic "false sharing"-like phenomenon, except here it is real sharing — every core wants to write to the same line.

Striped counters¶

Spread the counter across many cells; each core writes to its own cell most of the time. Reading the total requires summing the cells.

package striped

import (
    "runtime"
    "sync/atomic"
)

const cacheLine = 64

type Counter struct {
    cells []cell
}

type cell struct {
    val atomic.Int64
    _   [cacheLine - 8]byte // padding to fill a cache line
}

func New() *Counter {
    n := runtime.NumCPU()
    return &Counter{cells: make([]cell, n)}
}

func (c *Counter) Add(n int64) {
    idx := procPin() % len(c.cells)
    procUnpin()
    c.cells[idx].val.Add(n)
}

func (c *Counter) Load() int64 {
    var total int64
    for i := range c.cells {
        total += c.cells[i].val.Load()
    }
    return total
}

procPin is an internal runtime function that returns the current P's index. It is exposed via runtime/internal/sys or via tricks like runtime.GOMAXPROCS(0) and goid. In real code you might use a hash of goid or simply runtime.NumCPU() with cycling.

The padding is critical. Without it, multiple counter cells would share a cache line and false-share. The 64-byte padding ensures each cell sits alone on its line.

Performance¶

A striped counter on a 16-core machine with all cores writing: - Add: ~5-8 ns per call (each core hits its own line). - Load: ~50-100 ns per call (sum across 16 cells, each line cold to the reader).

This is a dramatic improvement over the naive counter under contention. The trade-off: Load is more expensive than a single-counter load.

When to stripe¶

• The counter is on a hot path with many concurrent writers.
• Reads are rare (you tally periodically for metrics, not on every operation).
• You can afford the extra memory: 64 bytes per cell × number of CPUs.

A typical request-rate counter for a 16-core service: 1024 bytes (16 cells × 64 bytes). Worth it for the contention savings.

metric/expvar and Prometheus counters¶

expvar.Int is just a single atomic counter; under contention it bottlenecks. Prometheus client libraries typically use one counter per metric, also single atomic.

For very high-rate counters (every request, every cache hit) consider wrapping the metric in a striped counter and flushing periodically to the metric library.

type StripedPromCounter struct {
    striped *Counter
    promCtr prometheus.Counter
}

func (s *StripedPromCounter) Inc() {
    s.striped.Add(1)
}

func (s *StripedPromCounter) Flush() {
    val := s.striped.Load()
    s.promCtr.Add(float64(val))
    // reset the striped counter... but this isn't trivial because Add is concurrent.
}

Resetting concurrent counters is the tricky part. A two-buffer scheme (write to A while reading from B, then swap) is a common pattern. We omit the implementation here; the principle is clear.

Sharding for Throughput¶

Sharding is the natural extension of striping: instead of one big limiter, use N smaller ones and route by hash.

Why shard a rate limiter¶

If you have a single limiter at 1M QPS, the mutex or atomic CAS will be your bottleneck. If you split into 16 shards each at 62.5k QPS, each shard has lower contention and the total throughput scales linearly with shards.

The cost: the rate guarantees become per-shard, not global. A shard might be at 100% utilisation while another is idle — the total admitted is below the global limit.

When sharding is safe¶

Sharding is safe when:

1. The limit is approximate. "About 100 RPS" is fine; "exactly 100 RPS" is not.
2. The traffic to each shard is balanced. Hashing by request hash gives good balance for unstructured traffic but not for traffic with hot keys.
3. The rate is high enough that the shard-level rate is meaningful. A limit of 1 RPS sharded into 16 shards gives each shard 1/16 RPS, which produces fractional tokens and weird behaviour.

Sharded token bucket¶

package shardedbucket

import (
    "hash/maphash"

    "golang.org/x/time/rate"
)

type ShardedLimiter struct {
    shards []*rate.Limiter
    seed   maphash.Seed
}

func New(numShards int, ratePerSec rate.Limit, burst int) *ShardedLimiter {
    s := &ShardedLimiter{
        shards: make([]*rate.Limiter, numShards),
        seed:   maphash.MakeSeed(),
    }
    perShardRate := ratePerSec / rate.Limit(numShards)
    perShardBurst := burst / numShards
    if perShardBurst < 1 {
        perShardBurst = 1
    }
    for i := range s.shards {
        s.shards[i] = rate.NewLimiter(perShardRate, perShardBurst)
    }
    return s
}

func (s *ShardedLimiter) Allow(key string) bool {
    var h maphash.Hash
    h.SetSeed(s.seed)
    h.WriteString(key)
    idx := int(h.Sum64() % uint64(len(s.shards)))
    return s.shards[idx].Allow()
}

The hash maps each key to a single shard. The shard's Allow is independent of the others, so contention is limited to the goroutines that share a shard.

Sharding for unkeyed limits¶

If your limit is global (no key), sharding still helps with contention by load-balancing goroutines to shards:

func (s *ShardedLimiter) AllowRandom() bool {
    // Use a per-goroutine or per-P random pick.
    idx := fastrand() % uint32(len(s.shards))
    return s.shards[idx].Allow()
}

fastrand is the runtime's per-P random source, accessible via runtime.fastrand (private, accessible via //go:linkname).

The randomness ensures load balance even when one goroutine makes many calls in a row. The total rate is approximately r even though no individual shard is at r.

Sharding and unfairness¶

Sharding introduces unfairness: two clients hashing to the same shard contend with each other, while a third client on a different shard does not.

For per-IP rate limiting this is usually fine — the IPs balanced across shards on average — but for premium customers who paid for guaranteed throughput, sharding is wrong: their request might land on a saturated shard and be denied while the shared budget is unspent.

The solution is per-customer limiters keyed by customer ID, with a tiered approach: a separate per-customer limiter for VIPs and a sharded shared limiter for everyone else.

Debouncer Deep Dive¶

We've spent a lot of time on throttling. Let's revisit debouncing with senior-level care.

The contract of a debouncer¶

A debouncer collapses a stream of events into a single trigger, fired after a period of silence. Three behaviours are usually relevant:

1. Trailing-edge debounce: fire after d of silence following the last event. This is the default.
2. Leading-edge debounce: fire immediately on the first event, then ignore subsequent events until d of silence.
3. Both-edge debounce: fire immediately on the first event, then fire again at the trailing edge if there were intervening events.

Most production debouncers are trailing-edge. Leading-edge is occasionally useful for "fire one notification per burst." Both-edge is rare.

Trailing debouncer¶

package debounce

import (
    "sync"
    "time"
)

type Trailing struct {
    mu     sync.Mutex
    timer  *time.Timer
    delay  time.Duration
    action func()
}

func NewTrailing(delay time.Duration, action func()) *Trailing {
    return &Trailing{delay: delay, action: action}
}

func (d *Trailing) Trigger() {
    d.mu.Lock()
    defer d.mu.Unlock()
    if d.timer != nil {
        d.timer.Stop()
    }
    d.timer = time.AfterFunc(d.delay, d.action)
}

func (d *Trailing) Stop() {
    d.mu.Lock()
    defer d.mu.Unlock()
    if d.timer != nil {
        d.timer.Stop()
    }
}

This is correct but allocates a fresh timer on every Trigger. For a high-rate event source the allocations matter.

Trailing debouncer with timer reuse¶

type ReusingTrailing struct {
    mu     sync.Mutex
    timer  *time.Timer
    delay  time.Duration
    action func()
}

func NewReusing(delay time.Duration, action func()) *ReusingTrailing {
    d := &ReusingTrailing{delay: delay, action: action}
    d.timer = time.AfterFunc(time.Hour, d.fire)
    d.timer.Stop()
    return d
}

func (d *ReusingTrailing) Trigger() {
    d.mu.Lock()
    defer d.mu.Unlock()
    d.timer.Reset(d.delay)
}

func (d *ReusingTrailing) fire() {
    d.action()
}

func (d *ReusingTrailing) Stop() {
    d.mu.Lock()
    defer d.mu.Unlock()
    d.timer.Stop()
}

time.Timer.Reset does not allocate; it reuses the existing runtime timer. On a hot debouncer this is a measurable saving.

A caveat: Reset after the timer has already fired is racy in Go versions before 1.23. The recommended pattern in older Go is to call Stop and drain the channel before Reset. Go 1.23+ made Reset safe in all cases by giving timers a single-element channel buffer with explicit "stop overwrites the slot" semantics.

Trailing debouncer with last-value semantics¶

A common variant: the debouncer fires the last value it saw, not just a trigger.

type ValueDebouncer[T any] struct {
    mu     sync.Mutex
    timer  *time.Timer
    delay  time.Duration
    value  T
    has    bool
    action func(T)
}

func NewValue[T any](delay time.Duration, action func(T)) *ValueDebouncer[T] {
    d := &ValueDebouncer[T]{delay: delay, action: action}
    d.timer = time.AfterFunc(time.Hour, d.fire)
    d.timer.Stop()
    return d
}

func (d *ValueDebouncer[T]) Trigger(v T) {
    d.mu.Lock()
    defer d.mu.Unlock()
    d.value = v
    d.has = true
    d.timer.Reset(d.delay)
}

func (d *ValueDebouncer[T]) fire() {
    d.mu.Lock()
    v, ok := d.value, d.has
    d.has = false
    var zero T
    d.value = zero
    d.mu.Unlock()
    if ok {
        d.action(v)
    }
}

The "last value wins" semantics is exactly right for cases like search input ("only search for the final query"), config reload ("apply the last config in the burst"), and UI state ("paint the final state").

Leading-edge debouncer¶

type Leading struct {
    mu      sync.Mutex
    nextOK  time.Time
    delay   time.Duration
    action  func()
}

func NewLeading(delay time.Duration, action func()) *Leading {
    return &Leading{delay: delay}
}

func (d *Leading) Trigger() {
    d.mu.Lock()
    now := time.Now()
    if now.Before(d.nextOK) {
        d.mu.Unlock()
        return
    }
    d.nextOK = now.Add(d.delay)
    d.mu.Unlock()
    d.action()
}

The leading-edge debouncer is just a "minimum interval" gate. It admits the first event, then drops everything until delay has passed. There is no timer needed at all.

Both-edge debouncer¶

type Both struct {
    mu      sync.Mutex
    timer   *time.Timer
    delay   time.Duration
    leading bool
    pending bool
    action  func()
}

func NewBoth(delay time.Duration, action func()) *Both {
    d := &Both{delay: delay, action: action}
    d.timer = time.AfterFunc(time.Hour, d.trailingFire)
    d.timer.Stop()
    return d
}

func (d *Both) Trigger() {
    d.mu.Lock()
    if !d.leading {
        d.leading = true
        d.timer.Reset(d.delay)
        d.mu.Unlock()
        d.action()
        return
    }
    d.pending = true
    d.timer.Reset(d.delay)
    d.mu.Unlock()
}

func (d *Both) trailingFire() {
    d.mu.Lock()
    fire := d.pending
    d.pending = false
    d.leading = false
    d.mu.Unlock()
    if fire {
        d.action()
    }
}