Clean Commits & Version-Control Hygiene — Optimize & Reconcile¶

Clean history is not just an aesthetic. The shape of the record — blob size, commit count, branch lifetime, atomicity — is a load-bearing input to clone time, CI pipeline duration, bisect resolution, review throughput, and merge-queue capacity. This file reconciles the human-facing discipline of clean commits with the machine-facing economics of operating a Git repository at scale. Each scenario states a concrete workflow, measures the cost in repo size / clone seconds / pipeline minutes, then resolves it on principle — keep blobs out of history, commit atomically, and let CI fetch only what it needs.

Table of Contents¶

1. A committed node_modules triples clone time
2. A deleted 400 MB binary is still in the clone
3. Generated artifacts in history defeat delta compression
4. CI clones full history on every job
5. A 12 GB monorepo nobody can clone in under 20 minutes
6. git log/git blame are slow because the commit-graph is missing
7. A repo-wide reformat poisons every git blame
8. git status takes 8 seconds on a working tree of 300k files
9. Kitchen-sink commits make git bisect useless
10. A 6-week feature branch costs 3 days to merge
11. Serialized merges throttle a 40-engineer team
12. A pre-commit hook lints the whole repo on every commit
13. Giant PRs stall review and inflate cycle time
14. Binary assets churn in history instead of Git LFS

Scenario 1 — A committed node_modules triples clone time¶

A front-end team committed node_modules/ "so the build is reproducible." The directory holds 240 MB across ~180,000 small files. The repo .git directory is 1.9 GB; a fresh git clone over a corporate VPN takes ~6 minutes and checks out 190,000 files.

Measurement / reasoning. The cost is twofold. Network: every byte of every version of every package is packed and transferred. Filesystem: checkout writes 180k inodes — on a spinning disk or a networked filesystem the checkout phase alone dominates. git count-objects -vH shows size-pack: 1.9 GiB; git verify-pack reveals the bulk is .js/.map text under node_modules. Worse, each npm install that changes a sub-dependency rewrites thousands of blobs, so history grows ~50 MB per dependency bump.

# .gitignore
node_modules/
dist/
*.tsbuildinfo

git rm -r --cached node_modules
git commit -m "build: stop tracking node_modules; rely on lockfile"

Scenario 2 — A deleted 400 MB binary is still in the clone¶

Six months ago someone committed assets/demo-build.dmg (400 MB), realized the mistake, and ran git rm demo-build.dmg in the next commit. The file no longer appears in the working tree or in HEAD. Yet new clones still transfer 400 MB and take 4× longer than the codebase justifies.

Measurement / reasoning. Git history is an append-only DAG of immutable objects. A git rm adds a new commit whose tree omits the file — it does not delete the blob, which remains reachable from the ancestor commit that introduced it. Every full clone walks all reachable objects, so the 400 MB blob ships forever. git rev-list --objects --all | git cat-file --batch-check | sort -k3 -n (or the tool below) surfaces it instantly:

git rev-list --objects --all \
  | git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' \
  | awk '/^blob/ {print $3, $4}' | sort -rn | head
# 419430400 assets/demo-build.dmg   <-- 400 MB still here

# Operate on a fresh mirror clone — this REWRITES every commit hash.
git clone --mirror git@host:org/repo.git repo-clean.git
cd repo-clean.git
git filter-repo --invert-paths --path assets/demo-build.dmg
git reflog expire --expire=now --all && git gc --prune=now --aggressive

Scenario 3 — Generated artifacts in history defeat delta compression¶

A backend repo commits the compiled bundle.min.js (1.2 MB) on every release. Over 300 releases that is, naively, 360 MB. The team assumes Git's delta compression will dedupe it. It does not, and .git is 280 MB.

Measurement / reasoning. Git's packfile delta compression works well on text that changes incrementally. Minified/bundled output is a different beast: a one-line source change can reorder the entire minifier output, shift every hash in the bundle, and produce a blob with almost no byte-level overlap with the prior version. The delta is nearly as large as the full object. The same pathology hits compiled binaries, encrypted files, and pre-compressed assets (.zip, .png, .jpg) — Git cannot delta two already-compressed streams. git verify-pack -v .git/objects/pack/*.idx | sort -k3 -rn shows these blobs storing full size, not deltas.

git lfs track "*.psd" "*.onnx" "*.png"
git add .gitattributes

# .gitattributes
*.psd  filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text

Scenario 4 — CI clones full history on every job¶

A CI pipeline runs lint, unit, integration, and build as four parallel jobs. Each job starts with a default git clone, pulling the full 800 MB history with 40,000 commits. Clone alone is ~70 seconds × 4 jobs × ~400 pipelines/day = roughly 31 hours of runner time per day spent cloning.

Measurement / reasoning. CI almost never needs history — it needs the tip of one ref to build and test. A full clone transfers every commit, every tree, every blob version ever. Time the difference:

time git clone               git@host:org/repo.git   # ~70s, 800 MB
time git clone --depth=1     git@host:org/repo.git   # ~6s,  60 MB

# CI config (provider-agnostic)
steps:
  - uses: checkout
    with:
      fetch-depth: 1   # shallow: tip commit only

Scenario 5 — A 12 GB monorepo nobody can clone in under 20 minutes¶

A 1,000-engineer monorepo has 12 GB of history, 2.1 million files at HEAD, and 600k commits. A cold clone takes 22 minutes; the working-tree checkout adds another 4. New hires lose half a morning; CI provisioning is brutal.

Measurement / reasoning. The total cost decomposes into: (a) downloading all historical blobs, (b) downloading the commit/tree graph, (c) writing 2.1M files to disk. Most developers touch a few directories and rarely need historical blob content — Git fetched it all anyway because the protocol historically required a complete object closure.

git clone --filter=blob:none git@host:org/repo.git

git sparse-checkout init --cone
git sparse-checkout set services/payments libs/common

scalar clone git@host:org/repo.git   # partial clone + sparse + commit-graph + maintenance, configured

Scenario 6 — git log/git blame are slow because the commit-graph is missing¶

On a 600k-commit repo, git log --oneline -20 -- some/file.go takes 5–8 seconds and git merge-base main feature is similarly sluggish. Engineers blame "the big repo," but the working set is small.

Measurement / reasoning. Without a commit-graph file, history-traversal commands must zlib-inflate and parse each commit object to learn its parents and dates — millions of small reads. Reachability queries (merge-base, --ancestry-path, bisect) cannot prune the DAG efficiently. The commit-graph stores parent pointers and generation numbers in a flat, mmap-able file, letting Git answer "is A an ancestor of B?" without walking the whole graph.

git config fetch.writeCommitGraph true
git commit-graph write --reachable --changed-paths
git config core.commitGraph true

git maintenance start   # schedules commit-graph, gc, prefetch, loose-object packing

Scenario 7 — A repo-wide reformat poisons every git blame¶

A team adopts an auto-formatter and lands one 90,000-line commit reformatting the whole codebase. Now git blame on almost any line points at "chore: apply prettier" by one engineer on one date. The real authorship and intent — the data blame exists to surface — is buried one layer down.

Measurement / reasoning. git blame attributes each line to the last commit that changed it. A mass-reformat changes every line, so it becomes the blame target everywhere. Engineers must now run git blame <sha>^ -- file repeatedly to dig past it — slow and error-prone. This is the version-control cost of a kitchen-sink commit, paid on every future investigation.

# .git-blame-ignore-revs  (committed to the repo root)
# Apply Prettier across the codebase
a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0

git config blame.ignoreRevsFile .git-blame-ignore-revs

Scenario 8 — git status takes 8 seconds on a working tree of 300k files¶

A monorepo checkout has 300,000 files. git status, git checkout, and even shell prompts that show branch state take 6–8 seconds, because each must lstat() every tracked path to detect changes.

Measurement / reasoning. Git's default change detection scans the entire working tree against the index — O(files) lstat syscalls per invocation. At 300k files on macOS/Windows (slower stat than Linux) this is several seconds, repeated dozens of times an hour. The OS already knows which files changed via its filesystem-event API; Git just wasn't asking.

git config core.fsmonitor true
git config core.untrackedCache true   # cache untracked-file scan results

Scenario 9 — Kitchen-sink commits make git bisect useless¶

A regression appears: a checkout total is off by a cent. The team reaches for git bisect to find the offending commit across 200 candidates. But the history is full of commits like "implement checkout v2" — each a 4,000-line mix of a feature, a refactor, formatting, and a config change. Bisect lands on one such commit. Now what?

Measurement / reasoning. git bisect is a binary search — log₂(N) steps to find the culprit. With 200 commits that is ~8 test runs, a beautiful speedup if each commit is a single logical change. When a commit bundles five unrelated changes, "this commit is bad" tells you almost nothing: you must manually dissect 4,000 lines to find which sliver caused the bug. The binary search succeeds in pinpointing a commit and then fails at its real job — pinpointing a change.

git bisect start HEAD v1.0
git bisect run ./scripts/check-checkout-total.sh   # exit 0 = good, non-0 = bad

Scenario 10 — A 6-week feature branch costs 3 days to merge¶

An engineer takes a "big rewrite" onto a branch and works it alone for six weeks. Meanwhile main advances 400 commits. When the branch finally opens a PR, the merge has 60 conflicting files, the rewrite assumes interfaces that have since changed, and reconciliation takes three engineer-days plus a tense review.

Measurement / reasoning. Merge cost grows super-linearly with branch age. The probability that branch and main touch overlapping code rises with both the branch's diff size and main's churn; conflict-resolution effort then scales with the product. A branch that diverges for 6 weeks against an active main accumulates merge debt the way an un-rebalanced loan accrues interest — and the interest is paid in a single painful lump at the end, exactly when context has decayed. Empirically, conflict count and resolution time both rise sharply past ~1 week of divergence on an active repo.

Scenario 11 — Serialized merges throttle a 40-engineer team¶

The team enforces "PR must be green against the latest main before merge." With 40 engineers landing ~50 PRs/day, each merge invalidates everyone else's "tested against main" status, so PRs must re-run CI and re-merge one at a time. CI takes 25 minutes. The integration pipeline becomes a single-lane bridge; PRs queue up and some authors re-base 4–5 times before landing.

Measurement / reasoning. Strict "rebase-and-retest before each merge" serializes integration: throughput is capped at one PR per CI cycle, i.e. 60/25 ≈ 2.4 merges/hour, ~19 over an 8-hour day — well under the 50 demanded. The backlog grows, and engineers burn time babysitting the queue. The bottleneck is serialization, not CI speed alone.

Scenario 12 — A pre-commit hook lints the whole repo on every commit¶

A team installs a pre-commit hook that runs ESLint + Prettier + type-check over the entire repository. On a medium repo this adds 40–90 seconds to every git commit. Developers start committing with --no-verify to escape it, so the hook protects nothing.

Measurement / reasoning. A commit changes a handful of files, but the hook processes thousands. Cost scales with repo size instead of changeset size — exactly backwards. The friction is so high it trains developers to bypass the hook, which is worse than having no hook (false sense of safety). The fix is to scope work to the staged set.

// package.json
"lint-staged": {
  "*.{ts,tsx}": ["eslint --fix", "prettier --write"],
  "*.{md,json}": ["prettier --write"]
}

# .pre-commit-config.yaml  (Python ecosystem)
repos:
  - repo: local
    hooks:
      - id: ruff
        entry: ruff check --fix
        language: system
        types: [python]   # framework passes only staged, matching files

Scenario 13 — Giant PRs stall review and inflate cycle time¶

A 2,800-line PR sits open for nine days. Reviewers keep deferring it ("need a 2-hour block"); when review finally happens it is shallow, the PR has drifted from main and needs re-testing, and the round-trip repeats. Meanwhile the author context-switches away and loses the thread.

Measurement / reasoning. Review quality and speed degrade non-linearly with diff size. Studies of review effectiveness (and most teams' own data) show defect-detection drops sharply past a few hundred changed lines — reviewers skim. A large PR is also a large merge target: longer open time means more main drift, more conflicts, and more re-runs of CI. Cycle time (open → merged) balloons, which directly suppresses deployment frequency.

Scenario 14 — Binary assets churn in history instead of Git LFS¶

A game/ML repo versions .fbx models, .png textures, and .onnx weights directly in Git. Each is 5–80 MB, and artists update them weekly. After a year the repo is 30 GB and a clone is a coffee-break-and-a-half. Branch checkouts that swap asset versions rewrite hundreds of MB on disk.

Measurement / reasoning. As in Scenario 3, binaries don't delta-compress, so every revision of every asset is stored in full and shipped in every full clone. Unlike build output, these assets are genuine source — they must be versioned. The cost isn't "don't commit them," it's "don't put their bytes in the main object store."

git lfs install
git lfs track "*.fbx" "*.png" "*.onnx"
git add .gitattributes

Rules of Thumb¶

• Blobs in history are forever. A git rm hides a file; only a history rewrite (git filter-repo) removes its clone-time cost. Prevent with .gitignore + a server-side size-limit hook; never rely on cleanup.
• Commit source, not derivatives. Lockfiles, not node_modules; build configs, not dist/. Anything reproducible by the build does not belong in the object store.
• Binaries don't delta-compress. Route versioned binary inputs to Git LFS; publish build outputs to an artifact registry.
• Shallow in CI, partial for big repos. --depth=1 for ephemeral jobs; --filter=blob:none + sparse-checkout + Scalar for large multi-team clones. Fetch the graph, defer the blobs.
• Turn on the free speed. git maintenance start gives you commit-graph, background gc, and prefetch; core.fsmonitor true makes status/checkout instant. Zero workflow change.
• Atomic commits pay compound interest. They make bisect pinpoint a change not a SHA, make revert surgical, and make review humane. One logical change per commit.
• Short branches, small PRs. Divergence cost is super-linear; integrate within a day. Use feature flags to decouple merge from release.
• Isolate mechanical changes (reformats, renames) into dedicated commits and add their SHAs to .git-blame-ignore-revs so archaeology stays fast and accurate.
• Scale hooks to the changeset, not the repo. Pre-commit runs only on staged files and only fast checks; heavy verification lives in CI. A slow hook gets --no-verify'd into uselessness.
• Merge queues beat serialized merges for busy teams — speculative batching lifts throughput while keeping main green.
• Measure before rewriting history. Rewrites are destructive and force re-clones; do them once, deliberately, and announce them.

Related Topics¶

• README.md — the positive rules of clean commits and version control this file optimizes against.
• find-bug.md — spotting non-atomic commits, kitchen-sink PRs, and committed artifacts before they land.
• professional.md — coordinating history rewrites and never force-pushing shared branches.
• Code Reviews — the reviewing side of small, atomic PRs; review latency vs. PR size.
• Refactoring — why isolating mechanical refactors into their own commits keeps blame and bisect clean.