Removing Secrets from Git History

Committing a password, API key, or private SSH key to a Git repository is one of the most common — and most dangerous — mistakes in software development. The instinct is to delete the file and commit again, but that only hides the secret from the current snapshot; Git’s entire history still contains it, byte for byte, and anyone who can read that history can recover it. This lesson explains why simply deleting a secret doesn’t remove it, how to actually rewrite history to purge it with tools like git filter-repo and BFG Repo-Cleaner, and — just as important — what you must do beyond Git to fully close the leak.

Overview: why “deleting” a secret isn’t enough

Every commit in Git is built from three kinds of objects, each addressed by the SHA-1 hash of its content: a blob stores the raw bytes of a single file, a tree is a directory listing that maps file names to blob (or subtree) hashes, and a commit points at exactly one tree — the complete snapshot of the repository at that moment — plus its parent commit(s), an author, and a message. A branch such as main is nothing more than a small file holding one commit hash; it moves forward every time you commit. HEAD normally points at a branch, which in turn points at a commit.

When you commit a secret in secrets.env, Git stores its contents in a new blob object and records that blob’s hash inside the tree for that commit. If you later run git rm secrets.env and commit again, Git creates a new tree for the new commit that no longer references the file — but the earlier commit, its tree, and the blob with the secret are untouched. They are still there in .git/objects, still reachable by walking backward from main through the parent chain, and still downloaded in full by anyone who clones the repository. Commands like git log --all or git show 2b6e9a1:secrets.env, or simply browsing the file history on GitHub, will surface it.

The only real fix is to rewrite every commit that ever contained the secret so that new blob, tree, and commit objects are generated without it, and then move the branch/tag refs to point at the new chain. The two mainstream tools for this are git filter-repo (the tool Git’s own documentation and GitHub now recommend) and the third-party BFG Repo-Cleaner. Both work by walking history and building a parallel set of objects; the old ones become unreachable (“dangling”) and are eventually deleted by garbage collection. You’ll also encounter git filter-branch in older tutorials — it does the same job but is officially deprecated in favor of filter-repo because it is far slower and easy to misuse.

Rewriting history is necessary but not sufficient. If the repository was ever pushed, cloned, or forked before the cleanup, every one of those copies still has the original commits with the secret in them — Git’s rewrite has no effect on repositories it doesn’t touch. GitHub’s internal caches, pull request diffs, and CI/CD logs can also retain the value independently. Treat any committed secret as permanently compromised: rotating or revoking the credential is the step that actually closes the leak, and it matters more than the history cleanup itself.

Syntax

The general forms of git filter-repo you’ll use for secret removal:

git filter-repo --path secrets.env --invert-paths
git filter-repo --replace-text replacements.txt
git filter-repo --force
  • --path — select a file or directory to act on (e.g. secrets.env).
  • --invert-paths — combined with --path, keep everything except the given path(s), i.e. delete that path from every commit.
  • --replace-text — read a file of old-string==>new-string rules (or plain strings to redact) and rewrite every blob that matches, anywhere in history.
  • --force — required when running filter-repo against a repository that isn’t a fresh clone made just for this purpose; it overrides the tool’s built-in safety check.

BFG Repo-Cleaner is a Java jar that solves the same two common cases with a simpler interface, operating on a bare mirror clone:

java -jar bfg.jar --delete-files id_rsa my-repo.git
java -jar bfg.jar --replace-text passwords.txt my-repo.git
  • --delete-files — remove any blob whose file name matches the given pattern from every commit.
  • --replace-text — redact matching strings inside file contents wherever they occur, keeping the rest of the file.

Examples

These build on each other: first proving a secret survives a normal delete, then removing a whole file from history, then redacting a single leaked value embedded in a file you still need.

Example 1: confirming the secret is still there

Suppose commit 7c4d0f8 added secrets.env with real credentials, and a later commit 2b6e9a1 deleted it. The file is gone from the working tree, but it’s still fully readable from history:

git log --all --oneline -- secrets.env
git show 2b6e9a1:secrets.env

Output:

2b6e9a1 feat: add deployment config
7c4d0f8 chore: initial commit
DB_PASSWORD=hunter2SuperSecret
API_KEY=sk_live_4eC39HqLyjWDarjtT1zdp7dc

The first command lists every commit that ever touched secrets.env, including the one that deleted it. The second checks out that one file exactly as it existed in the commit before the delete and prints its content — the secret, in full, even though the file no longer exists on main.

Example 2: purging a whole file from every commit

Once you know a file needs to disappear from history entirely, install git filter-repo (via pip install git-filter-repo or your package manager) and run it against a fresh clone:

git filter-repo --path secrets.env --invert-paths

Output:

Parsed 42 commits
New history written in 0.87 seconds; now repacking/cleaning...
Repacking your repo and cleaning out old unneeded objects
Completely finished after 1.35 seconds.

--invert-paths flips the selection so that, instead of keeping only secrets.env, filter-repo keeps everything except it. Every one of the 42 commits gets a rewritten tree and a new commit hash, cascading from the very first commit that touched the file all the way to the tip of every branch.

Example 3: redacting one leaked value inside a file you still need

Sometimes the secret isn’t a whole standalone file — it’s a hardcoded key sitting inside config/settings.py, a file you still need in the repository. Deleting the file isn’t an option, so redact the specific string instead. Create a replacements file:

sk_live_4eC39HqLyjWDarjtT1zdp7dc==>REMOVED

Then run:

git filter-repo --replace-text replacements.txt --force

Output:

Parsed 42 commits
New history written in 0.91 seconds; now repacking/cleaning...
Completely finished after 1.22 seconds.

Every blob in history containing the exact string sk_live_4eC39HqLyjWDarjtT1zdp7dc gets a new version with it replaced by REMOVED, and every commit whose tree referenced that blob is rewritten to point at the new one. --force was needed here because the working directory wasn’t a brand-new clone. After either rewrite, push the new history and rotate the credential:

git remote add origin git@github.com:example/my-repo.git
git push origin --force --all
git push origin --force --tags

filter-repo removes the origin remote by default as a safety measure, so it has to be re-added. Because every rewritten commit has a brand-new SHA, this has to be a real --force push — there is no shared ancestry for --force-with-lease to compare against.

How it works step by step

What git filter-repo actually does to the repository:

  1. It requires a repository that looks freshly cloned for this purpose (or the explicit --force override), so a botched rewrite can simply be discarded and re-cloned.
  2. It walks every commit reachable from any ref, oldest ancestor first.
  3. For each commit it inspects the tree and, per your filters, either drops matching blobs (--path --invert-paths) or rewrites matching byte sequences inside blobs (--replace-text), writing any changed blob out as a brand-new object with a new SHA-1.
  4. It builds a new tree object reflecting those blob changes.
  5. It creates a new commit object pointing at the new tree and at the already-rewritten version of its parent — so a change made deep in history cascades forward through every descendant commit, not just the one that literally contained the secret.
  6. Once the whole graph is rewritten, it moves every branch and tag ref to the tip of the new chain.
  7. The old commits, trees, and blobs are no longer referenced by anything, so they become dangling objects; filter-repo automatically repacks and expires them so they’re actually removed from disk rather than lingering until a manual git gc.

Because step 5 changes a commit’s parent hash, and a commit’s own hash is a function of its content including that parent hash, a single blob change near the root of history changes the SHA of literally every commit after it — this is why the rewrite always requires a full force-push rather than a normal one.

Common Mistakes

Mistake 1: deleting the file instead of rewriting history

git rm secrets.env
git commit -m "fix: remove secrets file"

This only affects the newest commit. Every earlier commit’s tree still points at the blob containing the secret, so it remains fully recoverable with git show or by cloning the repo. Fix: run git filter-repo --path secrets.env --invert-paths (or the BFG equivalent) so every historical commit is rewritten, then force-push.

Mistake 2: reaching for git filter-branch

git filter-branch --force --index-filter "git rm --cached --ignore-unmatch secrets.env" --prune-empty --tag-name-filter cat -- --all

This does the same job as filter-repo but re-checks out the whole repository once per commit, making it extremely slow on any repo of real size, and its flags are easy to get subtly wrong — forgetting --tag-name-filter cat, for instance, silently leaves tags pointing at the old history. Git’s own documentation recommends against it now. Fix: install git-filter-repo and use that instead.

Mistake 3: assuming the rewrite alone fixes the leak

Successfully purging a key from history and force-pushing feels like the job is done — but any collaborator’s existing clone, any fork, and any CI log that captured the value still has the original secret. Fix: rotate or revoke the credential (regenerate the API key, change the password, delete and re-issue the SSH key) immediately, regardless of how you clean up history.

Mistake 4: force-pushing a rewrite without warning collaborators

git push --force origin main

Every commit on the branch now has a different SHA than what teammates have locally. A plain git pull on their end doesn’t cleanly fast-forward — it tries to merge two unrelated histories and produces a tangle of duplicate commits and conflicts. Fix: announce the rewrite, and have collaborators either delete and re-clone the repository, or hard-reset their local branch to the new remote tip (git fetch then git reset --hard origin/main) instead of pulling.

Best Practices

  • Rotate or revoke the exposed credential first — a clean history rewrite does not undo an exposure that already happened.
  • Prefer git filter-repo over the deprecated git filter-branch; use BFG Repo-Cleaner when you just need to delete files or redact strings quickly.
  • Run the rewrite against a fresh git clone --mirror of the repository, not your everyday working copy, so a mistake can be discarded by re-cloning.
  • After force-pushing, tell every collaborator to re-clone or hard-reset — never just git pull — since the rewritten commits share no history with what they already have.
  • If the repository is or was public, contact GitHub Support to ask them to purge cached views and check for forks that may retain the old commits.
  • Prevent recurrence: keep real credentials in files listed in .gitignore, commit a .env.example with placeholder values instead of a real .env, and add secret-scanning to your workflow (a pre-commit hook or GitHub’s built-in secret scanning) so a credential-shaped string is blocked before it’s ever committed.
.env
*.pem
*.key
aws-credentials.json

Practice Exercises

  1. You committed aws-credentials.json with real AWS keys three commits ago on a shared main branch that two teammates have already pulled. Write out the full sequence of steps — Git commands and non-Git actions — you’d take to fully close this leak, in the right order.
  2. A hardcoded key sk_live_abc123xyz appears inside config/settings.py across 15 different commits, and you need to keep the rest of that file’s history. Which git filter-repo flag fits this case, and why is --path --invert-paths the wrong choice here?
  3. After you run git filter-repo and force-push, a teammate says git pull on their existing clone now shows dozens of unexpected conflicts and duplicate commits. Explain what happened and what they should do instead to get back in sync.

Summary

  • Deleting a file in a new commit does not remove it from Git’s history — every earlier commit’s tree still references the original blob, and it stays fully recoverable.
  • Purging a secret requires rewriting every commit that ever contained it, using git filter-repo or BFG Repo-Cleaner rather than the deprecated git filter-branch.
  • The rewrite changes the SHA of every affected commit, so it always requires a coordinated force-push and a fresh clone or hard reset by every collaborator.
  • History surgery does not undo an exposure by itself — always rotate or revoke the leaked credential.
  • --path --invert-paths removes whole files from history; --replace-text redacts specific strings while keeping the surrounding file.
  • Prevent repeats with .gitignore, a .env.example pattern, and pre-commit secret scanning.