Unit Tests — Optimize & Reconcile¶

A test suite that takes 12 minutes to run is, in practice, a suite that runs once a day. The F in FIRST — Fast — is not cosmetic: it is the load-bearing property that lets TDD work at all. This file reconciles the two forces that pull against each other in every test suite: trust (the suite catches real regressions and you believe its green) and speed (the suite is cheap enough that you run it on every save). Each scenario starts from a slow or untrustworthy suite, measures it, and resolves the tension with a principle — not a hack. The wrong optimizations buy speed by destroying trust (deleting assertions, sharing mutable fixtures, retrying flaky tests until green). The right ones buy speed and keep trust.

Table of Contents¶

1. The 8-minute suite that killed the TDD loop
2. t.Parallel exposes a shared-state bug (Go)
3. JUnit 5 parallel execution and the static singleton (Java)
4. pytest-xdist and the shared temp directory (Python)
5. Replacing a real database with an in-memory fake
6. The inverted pyramid: 90% of runtime in 5% of tests
7. Suite-level fixture vs per-test isolation (Python)
8. The flaky-retry tax
9. Profiling a slow suite to find the real cost
10. Test selection / impacted-tests / build caching
11. testcontainers vs in-memory: realism vs speed
12. CI sharding: 14 minutes to 2 minutes
13. sleep in tests and the time tax
14. bcrypt cost factor in the test config (Java)

Scenario 1 — The 8-minute suite that killed the TDD loop¶

A 1,400-test backend suite takes 8m 10s locally. The team writes tests, but nobody runs the suite before pushing — they wait for CI. TDD's red-green-refactor loop, which needs sub-10-second feedback to be usable, is dead. Bugs land in main and are caught hours later.

$ go test ./... -json | <tally by package>
unit (pure logic):        1,180 tests   →   9s
integration (real DB):      190 tests   →  6m 40s
e2e (HTTP + DB + queue):     30 tests   →  1m 21s

func TestOrderRepository_Integration(t *testing.T) {
    if testing.Short() {
        t.Skip("skipping integration test in -short mode")
    }
    // ... real DB ...
}

Scenario 2 — t.Parallel exposes a shared-state bug (Go)¶

A unit suite runs in 9s serially. Adding t.Parallel() to every test should cut it to ~2s on an 8-core machine. Instead, three tests now fail intermittently — sometimes all green, sometimes one red.

var counter int // package-level

func TestIncrement(t *testing.T) {
    t.Parallel()
    counter = 0
    increment()
    if counter != 1 { t.Fatalf("got %d", counter) }
}

func TestIncrementTwice(t *testing.T) {
    t.Parallel()
    counter = 0
    increment(); increment()
    if counter != 2 { t.Fatalf("got %d", counter) }
}

func TestIncrement(t *testing.T) {
    t.Parallel()
    c := &Counter{}      // local instance, owned by this test
    c.Increment()
    if c.Value() != 1 { t.Fatalf("got %d", c.Value()) }
}

Scenario 3 — JUnit 5 parallel execution and the static singleton (Java)¶

A 900-test JUnit 5 suite takes 3m 20s. Enabling parallel execution:

# junit-platform.properties
junit.jupiter.execution.parallel.enabled=true
junit.jupiter.execution.parallel.mode.default=concurrent

cuts it to 48s on a 12-core box — but PriceCalculatorTest and DiscountServiceTest now fail randomly.

public final class FeatureFlags {
    private static boolean PROMO_ENABLED = false; // process-wide static
    public static void setPromoEnabled(boolean v) { PROMO_ENABLED = v; }
}

@ResourceLock("FEATURE_FLAGS")
class PriceCalculatorTest { ... }

@ResourceLock("FEATURE_FLAGS")
class DiscountServiceTest { ... }

Scenario 4 — pytest-xdist and the shared temp directory (Python)¶

A 600-test pytest suite runs in 2m 50s. Adding xdist:

$ pytest -n auto      # spawns 8 workers

drops it to 34s — but test_export_report and test_import_report now fail ~1 run in 5.

def test_export_report():
    export("/tmp/report.csv")           # fixed path
    assert os.path.exists("/tmp/report.csv")

def test_export_report(tmp_path):
    target = tmp_path / "report.csv"
    export(str(target))
    assert target.exists()

@pytest.fixture(scope="session")
def db_url(worker_id):
    # worker_id is "gw0".."gw7" under xdist, "master" without it
    return f"postgresql://localhost/test_{worker_id}"

Scenario 5 — Replacing a real database with an in-memory fake¶

UserService tests each spin up a real Postgres connection, run a migration, insert rows, assert, and truncate. 80 such tests take 52s — 650ms each, almost all of it connection + migration overhead, not logic.

// Real interface the service depends on:
type UserRepo interface {
    Save(u User) error
    FindByEmail(e string) (User, bool)
}

// In-memory fake — a real implementation, not a mock:
type FakeUserRepo struct{ byEmail map[string]User }

func NewFakeUserRepo() *FakeUserRepo { return &FakeUserRepo{byEmail: map[string]User{}} }
func (r *FakeUserRepo) Save(u User) error      { r.byEmail[u.Email] = u; return nil }
func (r *FakeUserRepo) FindByEmail(e string) (User, bool) { u, ok := r.byEmail[e]; return u, ok }

func runRepoContract(t *testing.T, repo UserRepo) {
    require.NoError(t, repo.Save(User{Email: "a@x.com"}))
    _, ok := repo.FindByEmail("a@x.com")
    require.True(t, ok)
}
func TestFakeRepo_Contract(t *testing.T)  { runRepoContract(t, NewFakeUserRepo()) }
func TestRealRepo_Contract(t *testing.T)  { runRepoContract(t, newPostgresRepo(t)) } // -short skips

Scenario 6 — The inverted pyramid: 90% of runtime in 5% of tests¶

A suite has 2,000 tests / 11m. Profiling the tiers:

e2e (browser, full stack):    95 tests  →  9m 50s   (89% of runtime,  5% of tests)
integration:                 305 tests  →    55s
unit:                      1,600 tests  →    15s

Adding a feature requires touching 8 e2e tests; each local run costs 10 minutes. The suite shape is an ice-cream cone, not a pyramid.

e2e:           25 tests  →  2m 35s
integration:  305 tests  →     55s
unit:       1,670 tests  →     17s
total:                       ~3m 47s   (down from 11m)

Scenario 7 — Suite-level fixture vs per-test isolation (Python)¶

A pytest module has 40 tests; each rebuilds an expensive in-memory graph (parse a 4 MB schema file, build an index). Per-test, that setup costs 300ms → 12s of pure setup. Switching the fixture to scope="module" builds the graph once and shares it → 0.3s of setup. But now test_add_node fails when run after test_remove_node.

@pytest.fixture(scope="session")
def base_graph():
    return build_graph("schema_4mb.json")   # paid ONCE per session: 300ms

@pytest.fixture
def graph(base_graph):
    return base_graph.copy()                 # per-test: ~2ms shallow clone

Scenario 8 — The flaky-retry tax¶

CI flakes ~8% of runs. Someone adds automatic retries: pytest --reruns 3. Green-rate recovers, but the suite's wall-clock on a flaky run jumps from 4m to 11m, and developers have started ignoring failures ("just hit rerun").

Scenario 9 — Profiling a slow suite to find the real cost¶

A 700-test suite takes 6m. The team's instinct is "the database tests are slow, let's mock the DB." Before spending a week on that, profile.

$ pytest --durations=15
==== slowest 15 durations ====
58.0s  test_full_catalog_import       <- ONE test
41.0s  test_search_reindex
12.0s  test_pdf_generation
 0.9s  test_user_login
 ...

$ go test ./... -v 2>&1 | grep -E '^(ok|--- PASS)' | sort -k3 -h | tail
$ go test ./services -run TestSearch -cpuprofile=cpu.out && go tool pprof cpu.out

@RegisterExtension static TimingExtension timing = new TimingExtension(); // logs per-test ms
$ mvn test -Dtest=SlowSuite -Djdk.attach.allowAttachSelf=true # attach JFR / async-profiler

Scenario 10 — Test selection / impacted-tests / build caching¶

A monorepo runs all 12,000 tests on every PR — 22m — even when the PR only touches one leaf package's README or one service's pricing logic. Developers wait 22 minutes to validate a one-line change.

$ go test ./...        # first run
$ go test ./...        # second run, no changes:
ok   example.com/pricing   (cached)
ok   example.com/orders    (cached)

$ nx affected --target=test --base=main
   Running tests for 2 of 47 projects affected by changes...

Scenario 11 — testcontainers vs in-memory: realism vs speed¶

The team replaced their Postgres integration tests with H2 (in-memory SQL) "for speed." Tests dropped from 90s to 8s — but a production bug shipped: a query using Postgres JSONB operators and ON CONFLICT ... DO UPDATE that H2 silently accepted with different semantics. The fast tests were fast and wrong.

@Testcontainers
class OrderQueryIT {
    @Container
    static PostgreSQLContainer<?> pg = new PostgreSQLContainer<>("postgres:16")
            .withReuse(true);          // reuse the container across runs/classes

    @Test void jsonbContainmentQuery() { /* real JSONB, real ON CONFLICT */ }
}

Scenario 12 — CI sharding: 14 minutes to 2 minutes¶

After all local optimizations, the full CI suite is an honest 14m of genuinely necessary work (a real integration tier that can't be faked away). The team runs it on one CI machine. PRs queue.

# GitHub Actions, 8 shards
strategy:
  matrix:
    shard: [1, 2, 3, 4, 5, 6, 7, 8]
steps:
  - run: go test ./... -shard=${{ matrix.shard }}/8

Scenario 13 — sleep in tests and the time tax¶

A suite of 60 async/timing tests contains lines like time.sleep(2) / Thread.sleep(2000) / time.Sleep(2 * time.Second) to "wait for the background job to finish." The suite spends ~2 minutes asleep, and the tests still flake on slow CI machines.

// Bad: time.Sleep(2 * time.Second); assert(done)
// Good: poll up to a generous ceiling, return as soon as condition holds
require.Eventually(t, func() bool { return job.IsDone() }, 2*time.Second, 5*time.Millisecond)

type Clock interface{ Now() time.Time }

func TestTokenExpiry(t *testing.T) {
    clk := &FakeClock{t: base}
    tok := IssueToken(clk)
    clk.Advance(31 * time.Minute)     // instant
    require.True(t, tok.Expired(clk))
}

Scenario 14 — bcrypt cost factor in the test config (Java)¶

UserServiceTest has 120 tests, each creating a user (which hashes a password with bcrypt). The production cost factor is 12. The auth-heavy tests take 48s — and the team is about to "mock the password encoder" to speed them up.

// test profile bean — real BCrypt, just a cheap cost factor
@Bean @Profile("test")
PasswordEncoder testPasswordEncoder() {
    return new BCryptPasswordEncoder(4);   // ~4ms/hash vs ~250ms at 12
}

Rules of Thumb¶

• Fast is a property of the inner-loop suite, not of every test. Partition tiers (unit / integration / e2e) so the TDD loop stays sub-10s while slower tiers still run on push and in CI. (Scenario 1)
• Parallelism is a test for isolation, not a cause of bugs. If t.Parallel / JUnit concurrency / xdist breaks a test, the test was already sharing mutable state. Fix the isolation; never disable the parallelism. Run -race permanently. (Scenarios 2-4)
• Declare shared resources explicitly (@ResourceLock, per-worker worker_id, tmp_path) so the framework parallelizes everything else. (Scenarios 3-4)
• Prefer a fake (working in-memory implementation) over a mock for infrastructure, and pin it with a contract test run against both fake and real, plus a thin slice of real-integration tests. A fake without a contract test is a silent divergence waiting to ship. (Scenario 5)
• The pyramid is a performance strategy. Push logic verification down; reserve e2e for what only e2e can catch. A misplaced e2e test pays a ~600× runtime tax and gives worse failure messages. (Scenario 6)
• Share fixtures only when read-only; isolate when mutable. Build expensive state once (session scope), hand each test a cheap isolated copy. Never fix order-dependence by ordering tests. (Scenario 7)
• A flaky test is worse than no test. Track flake rate, quarantine, root-cause. Auto-retry is a thermometer to log, not a cure to forget. Drive the retry budget to zero by fixing causes. (Scenario 8)
• Profile before optimizing. Suite runtime is Pareto-distributed; --durations / pprof / JFR name the few tests that dominate. A day on the top 5 beats a week of "mock everything." (Scenario 9)
• The cheapest test is one you correctly don't run. Build caching (Go/Bazel/Nx) skips unchanged work; impacted-selection skips unaffected work — but only if the cache key and dep graph are honest. Always run the full suite on merge as a backstop. (Scenario 10)
• Match fidelity to what the test claims to verify. Logic test → fast fake. SQL test → real engine (Testcontainers, not H2). The low-fidelity-double-for-high-fidelity-claim is counterfeit trust. (Scenario 11)
• Shard by historical runtime, not test count, and only after removing waste — sharding attacks wall-clock by spending CPU-seconds concurrently, not by reducing them. (Scenario 12)
• Replace sleep with poll-until-condition; replace clock-dependent logic with an injected clock. Wall-clock time is shared global state and a fixed cost. (Scenario 13)
• Tune the cost knob, not the algorithm. For deliberately expensive operations (bcrypt, KDFs, backoff), lower the cost parameter in the test profile while keeping the real algorithm in the loop. (Scenario 14)

Related Topics¶

• README — the positive rules of clean unit tests (FIRST, one-assert-per-concept, behavior over implementation).
• find-bug — diagnosing the specific failure modes (flaky categories, order dependence, shared state) that the optimizations here resolve at the root.
• professional — senior-level judgment on when fast-but-lower-fidelity is the right call and when it is malpractice.
• Boundaries — isolating third-party engines, globals, and the clock behind injectable seams, which is what makes fakes, fake clocks, and parallelism possible.
• Refactoring — Move Method and Extract Class instincts applied to where an assertion lives (push it to the cheapest tool that can make it).

Related skills: unit-testing-patterns, mocking-strategies, test-data-management, integration-testing, profiling-techniques, property-based-testing.