Debounce and Throttle — Professional Level¶

Table of Contents¶

1. Introduction
2. HTTP 429 and the Standard Rate-Limit Headers
3. Retry-After Semantics and Client Behavior
4. Designing the Public API Contract
5. Per-User, Per-IP, Per-Endpoint Limits
6. Burst-Friendly vs Strict Limits
7. Server Throttling Under Load
8. Client-Side Throttling with Jitter
9. UI Event Debouncing — End to End
10. Search-as-You-Type
11. Autosave Debouncers
12. Resize, Scroll, Drag Throttling
13. Log Throttling
14. Metric Throttling and Aggregation
15. Alert Throttling
16. Email and Push Notification Throttling
17. Backpressure as Throttling
18. Integrating with Circuit Breakers
19. Integrating with Retry Policies
20. Service Mesh Rate Limiters
21. Edge Rate Limiting (CDN/WAF)
22. Observability Dashboards
23. Postmortem 1: The Cascading Login Storm
24. Postmortem 2: The Debouncer That Never Fired
25. Postmortem 3: The Misconfigured 429
26. Postmortem 4: Log Storm Fills the Disk
27. Postmortem 5: The Limiter That Lied
28. Common Anti-Patterns
29. Operational Playbooks
30. Self-Assessment
31. Summary

Introduction¶

The previous documents covered what rate limiters are, how to use them, and how to build them from scratch. This document covers what happens when they meet reality: a public API, a UI fielded by millions of users, a log pipeline under sudden load, an integration with three downstream services each with its own quota, a postmortem at 3 AM after the rate limiter caused or failed to prevent an outage.

The skills here are operational. They are the difference between a service that survives a bad day and one that becomes the bad day. They include the precise wire format of HTTP rate-limit responses, the timing math behind UI debouncers in real browsers, the architecture of a log throttler that does not lose critical signal, the dashboards and alerts that tell you a limiter is misbehaving, and the post-incident discipline of asking "what would have prevented this?" honestly.

Throughout we will draw on patterns from real systems: AWS, GCP, GitHub, Stripe, Slack, Cloudflare, and the small services where most of us actually work. The pattern language is shared; the engineering judgement is what separates a working system from a working-on-a-good-day system.

HTTP 429 and the Standard Rate-Limit Headers¶

The HTTP response code for rate limiting is 429 Too Many Requests, defined in RFC 6585. The status code is the only universally honored signal; everything else is convention.

The minimum compliant response¶

HTTP/1.1 429 Too Many Requests
Content-Type: application/json

{
  "error": "rate_limited",
  "message": "Too many requests; try again in 30 seconds."
}

This is enough for a client that knows the API to handle it. It is not enough for generic clients (web browsers retrying scripts, monitoring tools, third-party SDKs). For those we need headers.

The Retry-After header¶

RFC 7231 defines Retry-After, which can be either:

• A number of seconds: Retry-After: 30
• An HTTP-date: Retry-After: Wed, 21 Oct 2025 07:28:00 GMT

Use the seconds form. It is universally supported, easy for clients to parse, and unambiguous.

HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: application/json

{"error":"rate_limited"}

Many HTTP client libraries automatically honor Retry-After. The Go net/http library does not, but most retry middleware (e.g., hashicorp/go-retryablehttp, cenkalti/backoff) does.

The X-RateLimit-* family¶

There is no official standard, but a de-facto convention emerged from Twitter, GitHub, and Stripe:

• X-RateLimit-Limit: the maximum requests in the window.
• X-RateLimit-Remaining: the requests left in the window.
• X-RateLimit-Reset: the Unix timestamp (or seconds) when the window resets.

Example:

HTTP/1.1 200 OK
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4998
X-RateLimit-Reset: 1700000000

These headers should appear on every response, not just 429s. Clients use them to throttle themselves proactively rather than waiting for a 429.

Variations across providers¶

• GitHub: X-RateLimit-Limit, -Remaining, -Reset (Unix epoch seconds), plus -Resource for the bucket name and -Used.
• Twitter (X): same family plus x-rate-limit-incident-id.
• Stripe: returns 429 with Retry-After but does not expose remaining; uses different per-user limits server-side.
• GCP: returns 429 with Retry-After and a JSON body. Headers are minimal.
• AWS: returns 503 or 400 with a specific error type rather than 429 in many services (legacy API behavior). DynamoDB returns ProvisionedThroughputExceededException.

The lack of standardization is annoying but real. Document your API's specific behavior.

The RateLimit RFC (IETF)¶

There is an in-progress IETF draft, draft-ietf-httpapi-ratelimit-headers, that proposes standardized headers:

• RateLimit-Limit: window size and limit, e.g., 100, 100;w=60.
• RateLimit-Remaining: requests left.
• RateLimit-Reset: seconds until reset.

The X- prefix is officially deprecated by RFC 6648. New APIs should prefer the prefix-less form, but support both for backward compatibility.

Implementation in Go¶

package ratelimit

import (
    "fmt"
    "net/http"
    "time"
)

type Decision struct {
    Allowed   bool
    Limit     int
    Remaining int
    ResetAt   time.Time
}

func WriteHeaders(w http.ResponseWriter, d Decision) {
    w.Header().Set("RateLimit-Limit", fmt.Sprintf("%d", d.Limit))
    w.Header().Set("RateLimit-Remaining", fmt.Sprintf("%d", d.Remaining))
    w.Header().Set("RateLimit-Reset", fmt.Sprintf("%d", int(time.Until(d.ResetAt).Seconds())))
    w.Header().Set("X-RateLimit-Limit", fmt.Sprintf("%d", d.Limit))
    w.Header().Set("X-RateLimit-Remaining", fmt.Sprintf("%d", d.Remaining))
    w.Header().Set("X-RateLimit-Reset", fmt.Sprintf("%d", d.ResetAt.Unix()))
}

func Deny(w http.ResponseWriter, d Decision) {
    WriteHeaders(w, d)
    retryAfter := int(time.Until(d.ResetAt).Seconds())
    if retryAfter < 1 {
        retryAfter = 1
    }
    w.Header().Set("Retry-After", fmt.Sprintf("%d", retryAfter))
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusTooManyRequests)
    fmt.Fprintf(w, `{"error":"rate_limited","retry_after":%d}`, retryAfter)
}

The middleware:

func Middleware(l Limiter) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            key := keyFromRequest(r)
            d := l.Check(r.Context(), key)
            if !d.Allowed {
                Deny(w, d)
                return
            }
            WriteHeaders(w, d)
            next.ServeHTTP(w, r)
        })
    }
}

The headers go on every response. The deny path adds Retry-After and sends 429.

Edge cases¶

• Retry-After: 0 or negative: invalid. Always send at least 1 second. Many clients interpret 0 as "retry immediately" — which will likely fail again, causing a tight retry loop.
• Retry-After too long: clients may give up. Cap at a reasonable maximum (often 60 seconds for public APIs).
• Missing Content-Length: some buggy clients require it on 429s. Always write the body.
• CORS preflight 429: OPTIONS preflight requests should usually not be rate-limited. If they are limited, the browser will not send the actual request and the developer sees a CORS error, not a rate-limit error. Confusing.

A real-world response¶

GitHub's API on a rate-limit hit:

HTTP/2 403 Forbidden
date: Sun, 14 May 2026 12:00:00 GMT
content-type: application/json; charset=utf-8
x-ratelimit-limit: 60
x-ratelimit-remaining: 0
x-ratelimit-reset: 1716000000
x-ratelimit-used: 60
x-ratelimit-resource: core
retry-after: 60
{
  "message": "API rate limit exceeded for 203.0.113.10.",
  "documentation_url": "https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"
}

GitHub uses 403 instead of 429 for historical reasons. Modern APIs should use 429.

Retry-After Semantics and Client Behavior¶

The server tells the client when to retry. The client must honor it.

What "retry" means¶

A retry is sending the same request again. The intent: the next attempt should succeed because the rate-limit window has reset.

This is only safe for idempotent operations. A non-idempotent operation (POST creating a resource) cannot blindly be retried — you might create the resource twice.

The Go HTTP client does not retry by default. You add retry logic explicitly.

package retry

import (
    "context"
    "net/http"
    "strconv"
    "time"
)

type RetryClient struct {
    Client  *http.Client
    Max     int
    Backoff func(attempt int) time.Duration
}

func (c *RetryClient) Do(ctx context.Context, req *http.Request) (*http.Response, error) {
    for attempt := 0; attempt < c.Max; attempt++ {
        if attempt > 0 {
            select {
            case <-ctx.Done():
                return nil, ctx.Err()
            case <-time.After(c.Backoff(attempt)):
            }
        }
        resp, err := c.Client.Do(req.Clone(ctx))
        if err != nil {
            continue // network error: retry
        }
        if resp.StatusCode != 429 && resp.StatusCode != 503 {
            return resp, nil
        }
        wait := parseRetryAfter(resp.Header.Get("Retry-After"))
        if wait == 0 {
            wait = c.Backoff(attempt)
        }
        resp.Body.Close()
        select {
        case <-ctx.Done():
            return nil, ctx.Err()
        case <-time.After(wait):
        }
    }
    return nil, fmt.Errorf("retry limit exceeded")
}

func parseRetryAfter(h string) time.Duration {
    if h == "" {
        return 0
    }
    if secs, err := strconv.Atoi(h); err == nil {
        return time.Duration(secs) * time.Second
    }
    if t, err := http.ParseTime(h); err == nil {
        return time.Until(t)
    }
    return 0
}

This is a basic retry client that honors Retry-After. Real production clients are more sophisticated:

• Exponential backoff with jitter for transient network errors.
• Distinguish between retryable and non-retryable status codes.
• Avoid retrying non-idempotent methods unless explicitly allowed.
• Track cumulative wait time and give up gracefully.

The "thundering herd" problem in retries¶

Suppose a server returns Retry-After: 30 to a thousand clients simultaneously. All thousand wait exactly 30 seconds. At second 30 they all retry at once — and the server, having just recovered, is hit by another wall of requests and immediately returns 429 again.

This is the thundering herd. The fix: add jitter to the retry wait.

import "math/rand"

func jittered(base time.Duration) time.Duration {
    // Add 0-50% jitter.
    j := time.Duration(rand.Int63n(int64(base) / 2))
    return base + j
}

When honoring Retry-After, spread the retries over a window slightly larger than the suggested wait. The server sees a smooth ramp instead of a wall.

Backoff strategies¶

• Linear: wait = base * attempt. Predictable, gentle.
• Exponential: wait = base * 2^attempt. Aggressive backoff, useful when the failure is likely persistent.
• Decorrelated jitter: wait = random(base, prev_wait * 3). AWS-recommended, scales well across many clients.
• Exponential with full jitter: wait = random(0, base * 2^attempt). Simplest jittered backoff that prevents synchronization.

For a typical retryable client:

func decorrelatedJitter(attempt int, base, max time.Duration, prev time.Duration) time.Duration {
    upper := prev * 3
    if upper > max {
        upper = max
    }
    if upper < base {
        upper = base
    }
    return base + time.Duration(rand.Int63n(int64(upper-base)))
}

This is what golang.org/x/time/rate does not do — but every robust HTTP client must.

Idempotency keys for safe retries¶

For non-idempotent operations, send an Idempotency-Key header. Stripe pioneered this:

POST /v1/charges
Idempotency-Key: 7e8a...4f2c
{ "amount": 1000, "currency": "usd" }

The server stores the key and the response. If the client retries with the same key, the server returns the stored response (not re-executing the operation).

This makes any operation safely retryable. It is the right pattern for any external-facing POST.

Implementing on the server:

type IdempotencyStore interface {
    Get(ctx context.Context, key string) (resp []byte, ok bool, err error)
    Set(ctx context.Context, key string, resp []byte, ttl time.Duration) error
}

func IdempotencyMiddleware(store IdempotencyStore) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            key := r.Header.Get("Idempotency-Key")
            if key == "" {
                next.ServeHTTP(w, r)
                return
            }
            if resp, ok, _ := store.Get(r.Context(), key); ok {
                w.WriteHeader(http.StatusOK)
                w.Write(resp)
                return
            }
            rec := &recordingWriter{ResponseWriter: w}
            next.ServeHTTP(rec, r)
            if rec.status == http.StatusOK {
                store.Set(r.Context(), key, rec.body.Bytes(), 24*time.Hour)
            }
        })
    }
}

The store is typically Redis or a database table.

Cumulative timeout¶

A retry policy without a maximum total time can hang the caller indefinitely. Set a top-level deadline.

ctx, cancel := context.WithTimeout(parentCtx, 30*time.Second)
defer cancel()
resp, err := retryClient.Do(ctx, req)

The context is the ultimate retry budget. When it expires the client gives up, regardless of how many retries are left.

Designing the Public API Contract¶

If you offer a public API, the rate limit is a contract. Document it, advertise it, and stick to it.

Tiers¶

Most APIs offer tiers:

• Free: low limit, e.g., 60 requests/hour.
• Authenticated: medium, e.g., 5000/hour.
• Premium: high, e.g., 50000/hour.
• Enterprise: negotiable per-customer limits.

The tier is determined by the auth credentials. The limiter looks up the tier and applies the corresponding policy.

type Tier struct {
    Name           string
    RequestsPerHour int
    BurstSize       int
}

var tiers = map[string]Tier{
    "free":       {Name: "free", RequestsPerHour: 60, BurstSize: 10},
    "authed":     {Name: "authed", RequestsPerHour: 5000, BurstSize: 100},
    "premium":    {Name: "premium", RequestsPerHour: 50000, BurstSize: 500},
    "enterprise": {Name: "enterprise", RequestsPerHour: 500000, BurstSize: 5000},
}

func tierFromAuth(authHeader string) Tier {
    // ... decode JWT, look up plan, return tier
    return tiers["free"]
}

Documenting the limits¶

Your API docs must specify:

1. The numeric limit for each tier.
2. The window (per hour, per minute, per day).
3. The algorithm (token bucket with burst, sliding window).
4. The response code and headers when limited.
5. The Retry-After behavior.
6. The cost of different endpoints (some weigh more than others).

Example documentation snippet:

## Rate limits

Each authenticated request is subject to a rate limit based on your account tier.

| Tier         | Requests/Hour | Burst |
|--------------|--------------:|------:|
| Free         |            60 |    10 |
| Authenticated |          5000 |   100 |
| Premium      |         50000 |   500 |
| Enterprise   |  custom (contact sales) | |

The limit is enforced with a token bucket. Bursts up to the listed value are
allowed; the rate fills over the remaining time of the hour. When you exceed
your limit you will receive a `429 Too Many Requests` response with a `Retry-After`
header indicating when to retry.

Endpoints cost 1 token unless noted otherwise:
- `/search/*`: 5 tokens per request
- `/graphql`: variable cost based on query depth (see GraphQL limits)
- `/uploads`: 10 tokens per request, plus 1 token per MB

Response headers on every successful request:
- `X-RateLimit-Limit`: your hourly limit
- `X-RateLimit-Remaining`: tokens left this hour
- `X-RateLimit-Reset`: Unix timestamp when the hour resets

The clarity of the documentation is the API's UX. Lazy documentation produces angry developers.

Soft vs hard limits¶

A soft limit returns a warning header (e.g., X-RateLimit-Warning: nearing limit) but admits the request. A hard limit returns 429.

Many APIs use both: warn at 80% of the limit, deny at 100%. The warning gives clients time to slow down before hitting the wall.

func WriteHeadersWithWarning(w http.ResponseWriter, d Decision) {
    WriteHeaders(w, d)
    used := d.Limit - d.Remaining
    if float64(used)/float64(d.Limit) > 0.8 {
        w.Header().Set("X-RateLimit-Warning", "nearing limit")
    }
}

Cost-aware endpoints¶

A search query is more expensive than a single record fetch. Charge more tokens for expensive operations.

var endpointCost = map[string]int{
    "/search":         5,
    "/graphql":        10, // baseline; variable based on query
    "/uploads":        10,
    "/admin/audit":    20,
}

func costOf(r *http.Request) int {
    if c, ok := endpointCost[r.URL.Path]; ok {
        return c
    }
    return 1
}

func Middleware(l Limiter) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            key := keyFromRequest(r)
            cost := costOf(r)
            d := l.CheckN(r.Context(), key, cost)
            if !d.Allowed {
                Deny(w, d)
                return
            }
            WriteHeaders(w, d)
            next.ServeHTTP(w, r)
        })
    }
}

Cost-aware limiting is essential for GraphQL APIs where a single request can do enormous work. Compute the cost from the query AST.

Per-resource limits¶

In addition to a global per-user limit, you might enforce per-resource limits:

• 1000 messages per channel per minute.
• 100 commits per repository per hour.

The key includes the resource ID: ratelimit:user:U123:channel:C456.

This protects individual resources from being saturated by a single user.

When to skip rate limiting¶

Not every endpoint needs limiting:

• Health checks (/health, /ping): no limit; we want monitoring to always succeed.
• Admin endpoints behind auth: trust the auth; the limit becomes paranoia.
• Cached read-only endpoints: the cache absorbs the load; limiting is redundant.

Conversely, always limit:

• Authentication endpoints (/login, /signup): protect against credential stuffing.
• Resource-creation endpoints: prevent abuse.
• Search and aggregate endpoints: expensive on the backend.

Per-User, Per-IP, Per-Endpoint Limits¶

A real API has multiple keys layered together.

The key hierarchy¶

A request lookup might produce:

• user_id: from the auth token.
• api_key_id: also from the auth.
• ip: from the request.
• endpoint: from the URL path.

Each may have its own limit:

• Per-user: 1000/hour (the user has only so much quota).
• Per-API-key: 5000/hour (the key has its own quota; useful when one user has many keys).
• Per-IP: 100/hour (anonymous limit, defense against unauthenticated abuse).
• Per-endpoint per-user: 100/minute on /search (some endpoints are more expensive).

A request is admitted only if all applicable limits admit. The most restrictive wins.

func (m *MultiLimiter) Allow(r *http.Request) Decision {
    keys := m.deriveKeys(r)
    decisions := make([]Decision, 0, len(keys))
    for _, k := range keys {
        d := m.check(k)
        decisions = append(decisions, d)
        if !d.Allowed {
            // Reject; release the reservations already made.
            // (See hierarchical token bucket for proper handling.)
            return d
        }
    }
    // Commit all decisions.
    return decisions[len(decisions)-1] // or aggregate
}

IP-based limits and proxies¶

The "IP" in a per-IP limit is the client's IP, not necessarily r.RemoteAddr. Behind a load balancer or CDN, RemoteAddr is the proxy. The real IP is in X-Forwarded-For or X-Real-IP.

func clientIP(r *http.Request) string {
    if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
        // Take the first IP (the original client).
        if idx := strings.Index(xff, ","); idx >= 0 {
            return strings.TrimSpace(xff[:idx])
        }
        return strings.TrimSpace(xff)
    }
    if xr := r.Header.Get("X-Real-IP"); xr != "" {
        return xr
    }
    host, _, _ := net.SplitHostPort(r.RemoteAddr)
    return host
}

Caveats:

• X-Forwarded-For can be spoofed by clients. Trust it only when the request came through a known proxy.
• The first IP is the original client; subsequent IPs are intermediate proxies.
• IPv6 IPs may use shared address ranges. A /64 prefix is often the right key, since one client can have many IPv6 addresses.

NAT and shared IPs¶

Many users share a single NAT IP (corporate networks, mobile carriers). A per-IP limit penalizes all of them collectively.

Mitigations:

• Use authenticated keys when available; fall back to IP only for anonymous.
• Track per-IP in addition to per-user; a hard per-IP limit only kicks in for genuine abuse.
• Whitelist known shared NATs (corporate proxies, mobile carriers).

Per-endpoint limits¶

Different endpoints have different sensitivities.

• /auth/login: 5/minute per IP. Protect against credential stuffing.
• /auth/password-reset: 1/minute per email. Protect against email abuse.
• /api/search: 100/minute per user. Search is expensive.
• /api/messages: 1000/minute per channel. Messages are cheap but volume matters.

The per-endpoint limit is encoded in the route definition.

type RouteLimits struct {
    Path  string
    Limit int
    Per   time.Duration
    Key   func(*http.Request) string
}

var routes = []RouteLimits{
    {Path: "/auth/login", Limit: 5, Per: time.Minute, Key: ipKey},
    {Path: "/auth/password-reset", Limit: 1, Per: time.Minute, Key: emailKey},
    {Path: "/api/search", Limit: 100, Per: time.Minute, Key: userKey},
}

A router matches the path and applies the appropriate limit.

Burst-Friendly vs Strict Limits¶

The decision between burst-friendly (token bucket with large burst) and strict (small burst or sliding window) shapes the API's UX.

Burst-friendly¶

• Token bucket with b = r * 60 (burst = 60 seconds of rate).
• Allows clients to do legitimate startup bursts.
• Friendly to batch processing workloads.

Example: a user logs in, the client fetches their profile, inbox, settings, and recent activity in parallel. Five requests in one second. With burst-friendly limits, all five succeed and the user sees a snappy app.

Strict¶

• Sliding window with the limit equal to the long-term rate.
• No bursts allowed beyond the steady-state rate.
• Friendly to attack mitigation.

Example: a credential-stuffing attacker tries 100 username/password pairs in one second. With strict limits, only 1-2 succeed and the rest are denied immediately.

Mixed¶

Most APIs use both: burst-friendly for authenticated user actions, strict for anonymous or sensitive actions.

var limits = map[string]LimiterConfig{
    "user_actions":    {Algorithm: "token_bucket", Rate: 100/3600.0, Burst: 100}, // 100/hr, burst 100
    "login":           {Algorithm: "sliding_window", Rate: 5/60.0, Window: time.Minute},
    "password_reset":  {Algorithm: "sliding_window", Rate: 1/60.0, Window: time.Minute},
    "anonymous":       {Algorithm: "sliding_window", Rate: 10/60.0, Window: time.Minute},
}

Communicating burst to clients¶

X-RateLimit-* headers typically reflect the long-term limit. The burst is implicit.

If burst matters to client behavior (e.g., a client wants to know if it can send 20 requests right now), expose it:

X-RateLimit-Burst: 100

Few APIs do this, but it can be useful for SDK developers.

Server Throttling Under Load¶

A rate limiter prevents individual abusers from saturating the server. But what about the aggregate load — when all users are within their limits but the sum overwhelms the server?

Aggregate throttling¶

A global limiter caps total throughput regardless of per-user limits.

type AggregateLimiter struct {
    perUser *PerKeyLimiter
    global  *rate.Limiter
}

func (l *AggregateLimiter) Allow(r *http.Request) bool {
    if !l.perUser.Allow(userKey(r)) {
        return false
    }
    if !l.global.Allow() {
        return false
    }
    return true
}

The global limit is typically set just below the server's capacity (e.g., the max throughput the database can sustain). When the global limiter fires, every user sees occasional 429s — a sign that capacity is reached.

Load shedding¶

Beyond rate limiting, when the server is genuinely overloaded, drop some traffic to keep the rest fast. This is "load shedding."

Common approaches:

• Random drop: drop X% of requests when CPU/memory is high.
• Priority drop: drop low-priority requests first.
• Queue-length drop: drop requests when the queue exceeds a threshold.

Go example:

type Shedder struct {
    threshold float64 // CPU or queue-depth threshold
    sample    func() float64 // current load
}

func (s *Shedder) Allow() bool {
    load := s.sample()
    if load < s.threshold {
        return true
    }
    // Linearly drop more as load exceeds threshold.
    p := (load - s.threshold) / (1.0 - s.threshold)
    if p > 1.0 {
        p = 1.0
    }
    return rand.Float64() > p
}

When load is at 100% (saturation), drop 100% of new requests. When load is at threshold, drop 0%. Linear in between.

Adaptive shedding¶

A more sophisticated approach: shed by latency. Netflix's "adaptive concurrency limit" library is widely cited.

The idea: measure rolling p99 latency. If it exceeds a threshold, reduce the concurrency limit. If it stays below for a while, increase.

type AdaptiveConcurrencyLimiter struct {
    mu              sync.Mutex
    inFlight        atomic.Int64
    limit           int
    latencyP99      time.Duration
    targetLatency   time.Duration
    samples         []time.Duration
    lastAdjust      time.Time
}

func (l *AdaptiveConcurrencyLimiter) Allow() bool {
    cur := l.inFlight.Add(1)
    if int(cur) > l.limit {
        l.inFlight.Add(-1)
        return false
    }
    return true
}

func (l *AdaptiveConcurrencyLimiter) Done(latency time.Duration) {
    l.inFlight.Add(-1)
    l.mu.Lock()
    defer l.mu.Unlock()
    l.samples = append(l.samples, latency)
    if time.Since(l.lastAdjust) >= time.Second {
        l.adjust()
    }
}

func (l *AdaptiveConcurrencyLimiter) adjust() {
    sort.Slice(l.samples, func(i, j int) bool { return l.samples[i] < l.samples[j] })
    if len(l.samples) > 0 {
        l.latencyP99 = l.samples[len(l.samples)*99/100]
    }
    if l.latencyP99 > l.targetLatency {
        l.limit = int(float64(l.limit) * 0.9)
    } else {
        l.limit += 1
    }
    if l.limit < 1 {
        l.limit = 1
    }
    l.samples = l.samples[:0]
    l.lastAdjust = time.Now()
}

The limit shrinks under latency pressure and grows when the system is healthy. The result is automatic capacity sensing.

Priorities¶

Not all requests are equal. A logged-in paying user matters more than an anonymous one. A health check matters more than a regular request.

Tag each request with a priority and drop low-priority first:

const (
    PriorityCritical = iota // health checks, alerts
    PriorityHigh             // paid customers
    PriorityNormal           // regular users
    PriorityLow              // anonymous, batch jobs
)

func (s *PrioritizedShedder) Allow(p int) bool {
    threshold := s.thresholds[p] // higher priority -> higher threshold
    return s.currentLoad() < threshold
}

This is the architecture behind Google's "doorman" and many large APIs.

Client-Side Throttling with Jitter¶

When a client knows it has a rate limit, it should throttle itself rather than wait for the server to deny.

Client-side rate limiter¶

type Client struct {
    limiter *rate.Limiter
    http    *http.Client
}

func New(rps float64, burst int) *Client {
    return &Client{
        limiter: rate.NewLimiter(rate.Limit(rps), burst),
        http:    &http.Client{Timeout: 30 * time.Second},
    }
}

func (c *Client) Do(ctx context.Context, req *http.Request) (*http.Response, error) {
    if err := c.limiter.Wait(ctx); err != nil {
        return nil, err
    }
    return c.http.Do(req.WithContext(ctx))
}

This pre-throttles client requests. The server may also throttle, but the client tries to stay within its quota.

Learning the server's limit¶

If the server returns X-RateLimit-Limit headers, the client can dynamically tune its rate.

type LearningClient struct {
    Client
    mu sync.Mutex
}

func (c *LearningClient) Do(ctx context.Context, req *http.Request) (*http.Response, error) {
    if err := c.limiter.Wait(ctx); err != nil {
        return nil, err
    }
    resp, err := c.http.Do(req.WithContext(ctx))
    if err != nil {
        return nil, err
    }
    c.updateLimit(resp.Header)
    return resp, nil
}

func (c *LearningClient) updateLimit(h http.Header) {
    limit := h.Get("X-RateLimit-Limit")
    reset := h.Get("X-RateLimit-Reset")
    if limit == "" || reset == "" {
        return
    }
    l, _ := strconv.Atoi(limit)
    r, _ := strconv.ParseInt(reset, 10, 64)
    until := time.Until(time.Unix(r, 0)).Seconds()
    if until > 0 {
        rps := float64(l) / 3600 // assume hourly; could parse window
        c.mu.Lock()
        c.limiter.SetLimit(rate.Limit(rps))
        c.mu.Unlock()
    }
}

The client adapts as the server's policy changes.

Jittered scheduling¶

If many clients run the same scheduled job (e.g., "every hour fetch the latest data"), they synchronize and all hit the server at the top of the hour. Add jitter.

func scheduledFetch() {
    base := time.Hour
    jitter := time.Duration(rand.Int63n(int64(time.Minute)))
    for {
        time.Sleep(base + jitter)
        doFetch()
    }
}

A few seconds of jitter is usually enough to spread the load.

Adaptive client-side concurrency¶

The client maintains a semaphore for in-flight requests. The semaphore size adapts based on observed errors.

type AdaptiveClient struct {
    sema    chan struct{}
    maxSize atomic.Int64
}

func (c *AdaptiveClient) Do(ctx context.Context, req *http.Request) (*http.Response, error) {
    select {
    case c.sema <- struct{}{}:
    case <-ctx.Done():
        return nil, ctx.Err()
    }
    defer func() { <-c.sema }()
    resp, err := c.http.Do(req)
    if err != nil || resp.StatusCode == 429 {
        c.shrink()
    } else {
        c.grow()
    }
    return resp, err
}

The actual semaphore size adjustment via the channel buffer requires a non-trivial pattern (you can't resize a channel buffer directly in Go). One approach: use a counting semaphore (a golang.org/x/sync/semaphore.Weighted) which supports adjustable capacity.

UI Event Debouncing — End to End¶

Browsers fire events at very high rates. A keystroke handler runs on every key press; a scroll handler runs on every scroll position update; a resize handler runs continuously while the user drags a window edge. These cannot be processed inline at full rate.

The frontend pattern¶

In JavaScript, debouncing is universally implemented:

function debounce(fn, delay) {
  let timer;
  return function(...args) {
    clearTimeout(timer);
    timer = setTimeout(() => fn.apply(this, args), delay);
  };
}

const search = debounce((query) => {
  fetch(`/search?q=${query}`).then(handleResults);
}, 300);

document.getElementById('search').addEventListener('input', e => search(e.target.value));

300 ms is a typical debounce delay for search-as-you-type. The trade-off: longer feels laggy; shorter sends more requests.

Server-side complement¶

The frontend debounces. The backend should also rate-limit, because:

• A buggy client may bypass debouncing.
• A malicious client may deliberately spam.
• A widely-deployed client may not be updated even when the server's policy changes.

Defense in depth: trust the client to be polite, but enforce on the server.

Coordinated debouncing¶

For a multi-step UI (typing in one box, then clicking a button), the debouncer for typing should be flushed when the button is clicked. Otherwise the request from typing fires after the button click, possibly overwriting the user's intent.

const search = debounce(doSearch, 300);
const button = document.getElementById('submit');
button.addEventListener('click', () => {
  search.flush(); // fire pending debounced search now
  doSubmit();
});

A debouncer library with .flush() is essential. lodash's debounce supports this.

In Go on the server, our Debouncer from the previous document supports Flush().

Aborting in-flight requests¶

When the user types fast, you may have an in-flight request from a previous keystroke. The result of that request is now stale — but it may arrive after the response to the new keystroke and clobber the UI.

Solution: abort the previous request when starting a new one.

let abortCtrl;
const search = debounce((query) => {
  if (abortCtrl) abortCtrl.abort();
  abortCtrl = new AbortController();
  fetch(`/search?q=${query}`, { signal: abortCtrl.signal })
    .then(handleResults)
    .catch(err => { if (err.name !== 'AbortError') throw err; });
}, 300);

AbortController is the standard API. The aborted request's promise rejects with an AbortError, which we ignore.

On the server, the request's context is canceled when the client closes the connection (Go's http.Server propagates this automatically). The handler should respect cancellation:

func searchHandler(w http.ResponseWriter, r *http.Request) {
    rows, err := db.QueryContext(r.Context(), "SELECT ... WHERE ...")
    if err != nil {
        if errors.Is(err, context.Canceled) {
            return // client gave up; we should too
        }
        http.Error(w, err.Error(), 500)
        return
    }
    // ...
}

Last-result-wins¶

Even with abort, race conditions can leak through. A defensive pattern: tag each request with a sequence number; only render the result if the sequence number is the latest.

let lastSeq = 0;
async function search(query) {
  const seq = ++lastSeq;
  const result = await fetch(`/search?q=${query}`).then(r => r.json());
  if (seq === lastSeq) {
    render(result);
  }
}

The seq check ensures that if request 3 finishes before request 2, we still render only request 3.

This is more robust than abort because it does not rely on the abort actually working (some intermediaries don't cancel requests). It does cost the response data of the older requests — they are fetched but discarded.

Search-as-You-Type¶

The canonical UI debouncing use case. Let's design it end to end.

Goals¶

• The user types the quick brown fox. We want one search query (or maybe two — one when they pause after the quick, another when they finish).
• The result must appear within 100 ms of the user stopping typing.
• The server must not be overwhelmed by per-keystroke queries.

Frontend¶

const input = document.getElementById('search');
const results = document.getElementById('results');

let lastSeq = 0;
let abortCtrl;

const search = debounce(async (query) => {
  if (query.length < 2) {
    results.innerHTML = '';
    return;
  }
  if (abortCtrl) abortCtrl.abort();
  abortCtrl = new AbortController();
  const seq = ++lastSeq;
  try {
    const r = await fetch(`/search?q=${encodeURIComponent(query)}`, {
      signal: abortCtrl.signal
    });
    if (!r.ok) throw new Error('HTTP ' + r.status);
    const data = await r.json();
    if (seq === lastSeq) {
      render(data);
    }
  } catch (err) {
    if (err.name !== 'AbortError') {
      console.error(err);
    }
  }
}, 200);

input.addEventListener('input', e => search(e.target.value));

Key decisions:

• 200 ms debounce. Short enough to feel responsive, long enough to skip intermediate states.
• Minimum query length (2 chars) avoids the "every key on an empty box" pathology.
• Abort the previous request.
• Sequence-number guard against out-of-order responses.

Backend Go handler¶

func searchHandler(w http.ResponseWriter, r *http.Request) {
    q := strings.TrimSpace(r.URL.Query().Get("q"))
    if q == "" || len(q) < 2 {
        json.NewEncoder(w).Encode([]struct{}{})
        return
    }
    ctx, cancel := context.WithTimeout(r.Context(), 100*time.Millisecond)
    defer cancel()
    results, err := searchIndex.Query(ctx, q, 20)
    if err != nil {
        if errors.Is(err, context.DeadlineExceeded) {
            // Return partial or empty results rather than 500.
            json.NewEncoder(w).Encode([]struct{}{})
            return
        }
        http.Error(w, "search failed", 500)
        return
    }
    json.NewEncoder(w).Encode(results)
}

The handler enforces its own timeout. If the search index is slow, we return empty rather than a 500. This is a degradation choice — the user sees no results, but the system stays responsive.

Per-user rate limit¶

A user spamming keystrokes (deliberately or because of stuck keys) should not overwhelm the server.

var userLimiters sync.Map // user_id -> *rate.Limiter

func searchMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        userID := userIDFromCtx(r.Context())
        val, _ := userLimiters.LoadOrStore(userID, rate.NewLimiter(20, 5))
        l := val.(*rate.Limiter)
        if !l.Allow() {
            w.WriteHeader(http.StatusTooManyRequests)
            return
        }
        next.ServeHTTP(w, r)
    })
}

20 search QPS per user with burst 5. Very forgiving — accommodates fast typing without rate-limiting honest users.

Index-level throttling¶

The search backend (Elasticsearch, Algolia, custom index) has its own capacity. If the aggregate query rate exceeds it, the index degrades and all users see slowness.

var indexLimiter = rate.NewLimiter(5000, 1000) // 5000 QPS to the index

func (s *SearchService) Query(ctx context.Context, q string, limit int) ([]Result, error) {
    if !indexLimiter.Allow() {
        return nil, ErrIndexBusy
    }
    return s.realIndex.Query(ctx, q, limit)
}

When the index limiter denies, the caller can choose to return empty or to retry briefly. For search-as-you-type returning empty is usually fine — the user will type more and we'll try again.

Cache layer¶

A search cache absorbs repeat queries:

type CachedSearch struct {
    cache *lru.Cache[string, []Result]
    real  *SearchService
}

func (c *CachedSearch) Query(ctx context.Context, q string, limit int) ([]Result, error) {
    key := fmt.Sprintf("%s:%d", q, limit)
    if results, ok := c.cache.Get(key); ok {
        return results, nil
    }
    results, err := c.real.Query(ctx, q, limit)
    if err == nil {
        c.cache.Add(key, results)
    }
    return results, err
}

For search-as-you-type, the same query ("the quick ", "the quick b", "the quick br") appears in succession. The cache reduces index load dramatically.

Putting it together¶

The full stack:

1. Browser: 200 ms debounce, abort previous, sequence guard.
2. Network: cached at CDN/edge (if anonymous), per-user authentication.
3. Server middleware: per-user rate limit, sliding window.
4. Search service: in-process cache (LRU), index rate limit.
5. Index backend: own internal rate limit.

Defense in depth. Any one layer alone is insufficient; together they form a robust system.

Autosave Debouncers¶

A document editor (Google Docs, Notion, a code editor) autosaves the user's work without explicit save buttons. The naive implementation saves on every keystroke. The right implementation debounces.

Requirements¶

• The user types continuously. Don't save on every keystroke — too many writes to the database.
• When the user pauses, save the latest state within 1-2 seconds.
• If the user closes the tab, save immediately before unload.
• Failures must not lose data; retry with backoff.

A simple implementation¶

const saveButton = document.getElementById('save');
const editor = document.getElementById('editor');

let dirty = false;
let saveTimer;

editor.addEventListener('input', () => {
  dirty = true;
  clearTimeout(saveTimer);
  saveTimer = setTimeout(save, 1500);
});

window.addEventListener('beforeunload', (e) => {
  if (dirty) {
    save();
    e.preventDefault();
    e.returnValue = '';
  }
});

async function save() {
  if (!dirty) return;
  const content = editor.value;
  try {
    await fetch('/api/save', { method: 'POST', body: content });
    dirty = false;
  } catch (err) {
    console.error('save failed', err);
    saveTimer = setTimeout(save, 5000); // retry
  }
}

Key points:

• dirty flag tracks whether the document has unsaved changes.
• 1500 ms debounce is comfortable.
• beforeunload flushes pending changes.
• Failure schedules a retry.

Backend autosave handler¶

func saveHandler(w http.ResponseWriter, r *http.Request) {
    userID := userIDFromCtx(r.Context())
    docID := r.PathValue("doc")
    body, err := io.ReadAll(r.Body)
    if err != nil {
        http.Error(w, "read failed", 400)
        return
    }
    if len(body) > maxDocSize {
        http.Error(w, "too large", 413)
        return
    }
    if err := docStore.Save(r.Context(), userID, docID, body); err != nil {
        http.Error(w, "save failed", 500)
        return
    }
    w.WriteHeader(http.StatusNoContent)
}

The handler is straightforward. The interesting part is the per-document rate limit:

type DocLimiter struct {
    perDoc *PerKeyLimiter
}

func (l *DocLimiter) Allow(docID string) bool {
    return l.perDoc.Allow(docID)
}

We limit saves per document to (say) 10 per minute. A user editing one document doesn't need more than that; the autosave delay absorbs faster edits.

Optimistic concurrency¶

What if two users edit the same document? Last-write-wins loses data.

Solution: optimistic concurrency control. Each save includes a version number; the server rejects if the version is stale.

type SaveRequest struct {
    Content string
    Version int64
}

func (s *DocStore) Save(ctx context.Context, docID string, req SaveRequest) error {
    res, err := s.db.ExecContext(ctx, `
        UPDATE docs SET content = $1, version = version + 1, updated_at = NOW()
        WHERE id = $2 AND version = $3
    `, req.Content, docID, req.Version)
    if err != nil {
        return err
    }
    n, _ := res.RowsAffected()
    if n == 0 {
        return ErrVersionConflict
    }
    return nil
}

The client receives 409 Conflict on a version mismatch and must reconcile (typically by reloading and asking the user to redo their changes, or by merging if the editor is collaborative).

Operational transform / CRDT for real collab¶

For real-time collaboration (multiple users editing simultaneously), debounced saves are not enough. You need OT or CRDT. That's a separate topic. For this document we'll focus on single-user autosave.

Resumable saves¶

Large documents might fail mid-save. Resumable uploads (multipart with offsets) help.

For autosave the simpler approach is: keep a local cache of the document, save on a debounce, and on failure retry with the cached version. The local cache is the source of truth until saved.

Resize, Scroll, Drag Throttling¶

UI events that fire very rapidly need throttling, not debouncing.

Resize¶

The resize event fires continuously while the user drags a window edge. Without throttling, every layout calculation runs hundreds of times per second.

function throttle(fn, ms) {
  let last = 0;
  let pending;
  return function(...args) {
    const now = Date.now();
    if (now - last >= ms) {
      last = now;
      fn.apply(this, args);
    } else if (!pending) {
      pending = setTimeout(() => {
        last = Date.now();
        pending = null;
        fn.apply(this, args);
      }, ms - (now - last));
    }
  };
}

const onResize = throttle(() => {
  recalculateLayout();
}, 100);

window.addEventListener('resize', onResize);

Throttle to 10 fires per second (every 100 ms). Layout recalculation is now cheap.

requestAnimationFrame¶

For visual updates, requestAnimationFrame is the right primitive. It fires at the browser's repaint rate (usually 60 Hz). Use it instead of a fixed throttle.

let rafPending = false;
window.addEventListener('scroll', () => {
  if (rafPending) return;
  rafPending = true;
  requestAnimationFrame(() => {
    rafPending = false;
    updateScrollIndicator();
  });
});

Scroll updates that paint are tied to the screen refresh. Any more often is wasted; any less often is visibly choppy.

Drag¶

Drag events fire on every pointer move — up to thousands per second on a fast mouse with a high-polling-rate sensor. Throttle or use requestAnimationFrame.

const dragHandler = throttle((e) => {
  updatePosition(e.clientX, e.clientY);
}, 16); // ~60 fps

element.addEventListener('pointermove', dragHandler);