I Built an AI Code Review Skill That Thinks Like a Staff Engineer. Here is Every Technical Detail.

Most AI code review tools are just wrappers around a single prompt.

You paste a git diff, the model says "consider adding a null check here", and you close the tab. Nothing gets fixed. Senior engineers still spend half their week leaving the same comments on every pull request.

I wanted something different. Not a chatbot you query. A skill that runs automatically, knows your codebase history, has five specialized reviewers working in parallel, and an adversarial agent that tries to disprove every finding before it reaches you.

I just open-sourced the entire project on GitHub: 👉 github.com/atish23/squad-review

Here is how it works under the hood, file by file, script by script.

What Problem Are We Actually Solving?

Here is the real pain point.

Your senior engineers have been reviewing code for two or three years. They have caught the same bugs hundreds of times:

The database connection pool that blew up under load.
The missing null guard before a payment call.
The test scenario that was copy-pasted without changing the assertion.

That knowledge lives in their heads. Or buried in old PR threads nobody reads.

A generic AI review prompt does not have any of that context. It hallucinates. It flags things that are already handled in a sibling service. It writes three paragraphs about formatting when a linter runs in 2 seconds.

The goal here was simple: codify two senior engineers' review instincts into an automated skill that runs on every PR.

The Architecture at a Glance

  PR URL or ID
       |
       v
  [prepare.sh]       <-- fetches diff, threads, pipeline, builds annotated.txt
       |
  [Scope Gate]       <-- skips lanes not warranted by the diff
       |
  ┌────┴──────────────────────────────────────┐
  |      Parallel Lane Agents                 |
  |  code | design | tests | security | prec  |
  └────┬──────────────────────────────────────┘
       |
  [13-step Merge]   <-- dedupe, confidence gate, re-evidence blockers
       |
  [Verifier Agent]   <-- KEEP / DROP / DOWNGRADE_TO_PROBE per finding
       |
  [Chat Output]      <-- never auto-posted. You confirm each comment.

The File Structure

This is not a single prompt file. It is a structured skill bundle:

~/.claude/skills/squad-review/
  SKILL.md            # The orchestrator. Frontmatter triggers it.
  review-rules.md     # ~90 rule IDs across 14 categories
  voice.md            # Comment phrasing rules, measured on 400+ real comments
  mindmap.md          # Review order: hygiene -> naming -> structure -> security -> tests
  lane-contract.md    # What each lane receives, what it must return
  data/
    learnings.json        # Accepted, rejected, reworded comments over time
    squad-comments.json   # Source corpus: 400+ real review comments
    squad-threads.json    # Full thread context for those comments
    skill-proposals.md    # Auto-generated rule improvement proposals
    reviewed/             # One JSON per reviewed PR, keyed by repo-iid

~/.claude/agents/
  squad-review-code.md
  squad-review-design.md
  squad-review-tests.md
  squad-review-precedents.md
  squad-review-security.md
  squad-review-verifier.md

Every file has a single responsibility. Nothing is in SKILL.md that belongs in a script. Nothing is in a script that belongs in a rule.

The Scripts (11 of Them)

All deterministic work is handled by shell and Python scripts, not LLM prompt tokens:

scripts/
  prepare.sh          # The main setup script. Runs before any agent.
  fetch_mr.sh         # Fetches MR metadata, diffs, commits, pipeline status
  annotate_diff.py    # Turns raw diff into old|new| format for line anchoring
  precedents.py       # Matches current diff patterns against squad-comments.json
  notes_api.py        # Wrapper for Git host notes API calls
  post_comment.sh     # Posts a comment to a specific line with dry-run support
  edit_note.sh        # Edits an existing comment by note ID
  delete_note.sh      # Deletes a comment by note ID
  lint_skill.py       # Validates the whole skill: missing files, broken IDs, bad learnings
  refresh_comments.sh # Re-fetches the source corpus from Git host
  daily_learn.sh      # Runs daily, generates proposals in skill-proposals.md

prepare.sh is the workhorse. It does all the heavy lifting before a single agent runs.

Step 1: Prepare (Before Any Agent Touches the PR)

You trigger the review with /squad-review <args>.

Args can be a full PR URL, yourorg/your-repo#123, queue (open PRs where you are a reviewer), or repo:your-repo (all open non-draft PRs). Flags: --fast, --nits, --save, --brief.

prepare.sh runs first. It creates a temp working directory and builds these files:

File	What it contains
`mr.json`	PR metadata: title, description, author, branch, target
`commits.json`	Commit list with commit messages
`diffs.json`	Raw per-file diff content
`existing_threads.json`	All current threads: human, bot, resolved, outdated
`pipeline.json`	Latest CI pipeline status per job
`annotated.txt`	Diff in `old
`precedents.txt`	Past review comments on the same files, classes, packages
`context.json`	Head SHA, base SHA, ticket keys, local clone path
`paired_mrs.json`	Open PRs in related repos that share a ticket key

Ticket keys are extracted with a regex ([A-Z][A-Z0-9]+-\d+) across the PR title, branch name, description, and commit messages. Those keys drive the design lane later.

Step 2: Scope Gate (Don't Run What You Don't Need)

Running five agents on a one-line typo fix wastes tokens and takes longer than the fix.

The prepare script writes a file_signals block into context.json. The orchestrator reads it and skips lanes that are not warranted:

No ticket key and no spec link in description? Skip design.
No production, test, or database files changed? Skip tests.
Docs-only or test-only diff? Skip security.
code and precedents never skip.

The orchestrator can override any skip if the diff warrants it.

Step 3: Re-Review Caching Check

Before launching anything, the orchestrator checks data/reviewed/<repo>-<pr_iid>.json:

Same HEAD as last review? Show the previous findings. Stop. Do not re-run.
HEAD changed, known changed files? Keep old findings outside the changed files. Re-run only the relevant lanes on the new diff.
HEAD changed, changed files unknown? Full re-review.

This means you are never paying twice for the same work.

Step 4: The Five Lane Agents (Running in Parallel)

All five lanes get the same prompt: Work dir: <WORKDIR>. MR: <url>. Follow your agent definition and lane-contract.md. Return only the lane-contract output.

They all run at the same time. You wait for all of them before moving to the merge step.

Every lane returns the same shape:

{
  "LANE": "code",
  "STATUS": "ok",
  "FINDINGS": [
    {
      "anchor": "src/payments/processor.ts:L42",
      "rule": "ERR-003",
      "draft": "user can be null after find(). Add guard before .email.",
      "evidence": "grep result showing no null check at L41-43",
      "confidence": 92
    }
  ],
  "NOTES": "..."
}

Every lane also reads learnings.json before generating findings. If a rule was rejected by you in a past review, it does not raise that finding again.

Lane: code

Checks every rule category except tests, Cucumber, design, and security. That is ERR, DB, CFG, OBS, API, NAM, STR, DOM, XRP, and PRC rules. For cross-repo impact (XRP rules), it greps the default branches of sibling repos. Never their working trees.

Lane: design

Finds the product spec, architecture decision records, and design docs linked to this PR. If it can only reach the wiki cache (not the live page), every finding is capped at 79 confidence and written as a probe, not a directive.

Lane: tests

Builds a coverage map: which classes and methods have tests, and where the gaps are. Checks unit test rules (TST-001 to TST-003) and Cucumber scenario rules (BDD-001 to BDD-002).

Lane: precedents

Runs on a lighter model (haiku or flash). Matches patterns in the current diff against squad-comments.json, the source corpus of real comments. Findings are capped at 89 confidence unless the context is an exact match.

Lane: security

Assumes every external input is attacker-controlled. Checks:

PII field annotations
Authorization guards per method (@HasPermission, @PreAuthorize)
Object-level authorization (IDOR checks)
Input validation before processing
Secrets and credentials in code or config
TLS configuration

Step 5: The 13-Step Merge Engine

After all lanes finish, the orchestrator runs through these 13 steps in order:

Parse all lane outputs.
Drop anything marked rejected in learnings.json.
Treat precedent findings as hints, not blockers.
Re-run the evidence yourself for every blocker finding. If you cannot reproduce it with a grep or file read, drop it.
Dedupe across lanes: same anchor within 3 lines, or same root cause, counts as one finding.
Resolve conflicts between lanes.
Dedupe against existing PR threads, including bot threads and outdated ones.
Apply the confidence gate: drop below 60, downgrade 60-79 to probes.
Run the adversarial verifier.
Apply verifier verdicts.
Re-voice every draft.
Pick the PR verdict (approve, comment, request changes).
Save the full review state to data/reviewed/<repo>-<pr_iid>.json.

Step 6: The Adversarial Verifier (The Game Changer)

This is what separates this skill from every generic AI review tool.

After the merge step collects all findings, the verifier runs as a separate agent. Its job is to argue for the PR author.

For each finding, it asks:

Does the evidence actually reproduce when I run it?
Is this already handled upstream, in middleware, or in a sibling service?
Is this satisfied by a framework or type system guarantee?
Is this finding outside the diff scope entirely?
Was this already raised in an existing thread?
Was this pattern previously rejected in learnings.json?
Is this a nitpick dressed up as a blocker?

The verifier returns one of three verdicts per finding:

Verdict	What happens
`KEEP`	Finding survives. Must have confidence >= 80.
`DROP`	Finding is silently removed.
`DOWNGRADE_TO_PROBE`	Finding survives but becomes a question, not a directive.

A KEEP below 80 is not allowed. If the verifier is not sure, it drops it.

The Confidence Rubric

90-100  Reproduced with grep or file read. Impact is certain. Blocker.
80-89   Strong evidence, small doubt. Finding.
60-79   Probe only. Phrased as a question. Never a directive.
< 60    Drop silently. Never reported.

This rubric applies to every lane and to the verifier. It is the single most important mechanism that keeps the output clean.

The Voice Rules (Senior Engineer Voice)

The voice was measured from 400+ real review comments by two senior engineers on the same codebase:

Median comment length: 8 words.
75th percentile: 14 words.

Six allowed shapes:

Directive: extract <method> to its own service
Soft suggestion: Can we move this to the validator layer?
Probe: Is this safe if the response is empty?
Correction with fact: findById returns Optional, not null
Risk callout: this will deadlock if two threads hit it simultaneously
Pointer: align with pattern in RailsService.java:L88

Banned in every comment:

The word "Consider"
Any praise ("good approach", "nice work")
Severity labels in comment text (the label lives in metadata)
Emoji in comments
Any AI fingerprints ("I noticed", "It seems like", "You might want to")
Dead-code probes longer than 5 words (the grep result goes in evidence, not the comment)

The Learning Loop

Every comment you accept, reject, or reword gets logged to learnings.json:

{
  "date": "2026-10-16",
  "repo": "rails-service",
  "pr": "312",
  "rule": "ERR-003",
  "kind": "rejected",
  "before": "user can be null after find(). Add guard.",
  "after": null,
  "note": "null handled upstream in the repository layer"
}

The next time any lane or the verifier sees a pattern matching that rule in the same context, it will not raise it.

refresh_comments.sh and daily_learn.sh run on a daily schedule. They refresh the source corpus and write rule improvement proposals to skill-proposals.md. You review and merge those proposals. The skill proposes, you decide.

lint_skill.py validates the whole skill after any edit: missing files, broken rule ID references, duplicate IDs, malformed learnings entries. Run it every time you change anything.

Try It on Your Own Codebase

The entire repository is public and ready to clone:

# 1. Clone the repository
git clone https://github.com/atish23/squad-review.git
cd squad-review

# 2. Run the one-line installer
bash setup/install.sh

The installer copies the orchestrator, sub-agents, rules, and scripts directly into your Claude Code or Antigravity configuration.

Restart your CLI session, open any PR, and run:

/squad-review https://github.com/yourorg/your-repo/pull/123

Final Thoughts

"AI code review should not check syntax. It should bottle your senior engineers' best instincts, verify them against the actual code, and run them on every commit without burning anyone's afternoon."

Star the repo and check out the code: 👉 https://github.com/atish23/squad-review