Skip to content

PR Agent

Shane Neuville edited this page Apr 20, 2026 · 12 revisions

The PR Agent validates fixes through automated testing, explores alternatives using multiple AI models, and synthesizes everything into actionable recommendations.

%%{init: {'theme': 'dark', 'themeVariables': { 'primaryColor': '#1e1e2e', 'primaryTextColor': '#cdd6f4', 'primaryBorderColor': '#45475a', 'lineColor': '#6c7086', 'secondaryColor': '#313244', 'tertiaryColor': '#181825'}}}%%
flowchart LR
    subgraph gate ["🧪 GATE"]
        direction TB
        G1[Verify tests fail without fix]
        G2[Verify tests pass with fix]
        G1 --> G2
    end
    
    subgraph review ["🤖 PR REVIEW"]
        direction TB
        R1[Pre-Flight: gather context]
        R2[Try-Fix: 4 models explore alternatives]
        R3[Report: write recommendation]
        R1 --> R2 --> R3
    end
    
    subgraph post ["📊 POST"]
        direction TB
        P1[Post AI summary comment]
        P2[Apply agent labels]
        P1 --> P2
    end
    
    gate --> review --> post
    
    style gate fill:#1e1e2e,stroke:#89b4fa,stroke-width:2px,color:#cdd6f4
    style review fill:#1e1e2e,stroke:#cba6f7,stroke-width:2px,color:#cdd6f4
    style post fill:#1e1e2e,stroke:#a6e3a1,stroke-width:2px,color:#cdd6f4
Loading

Every fix is tested. The agent doesn't theorize—it implements each approach, runs tests, and reports what works.


Quick Start

Recommended: Use Plan Mode First

For the best results, start in plan mode to create and review a detailed plan before execution:

  1. Enter plan mode: Press Shift+Tab or use /plan
  2. Request a review plan:
    /plan review PR #12345 - create a detailed plan for the review
    
  3. Review the plan: Copilot will create a structured plan. Review the steps and make adjustments.
  4. Exit plan mode: Press Shift+Tab to switch back to execution mode
  5. Execute the plan:
    proceed with the plan
    

This workflow gives you control over the review approach before any actions are taken.

Direct Invocation (Alternative)

If you prefer to start immediately without planning:

# Start GitHub Copilot CLI
copilot

# Ask it to review a PR
please review PR #12345

Trigger Phrases

Phrase Description
"Fix issue #XXXXX" Works whether or not a PR exists
"Work on issue #XXXXX" Investigate and implement a fix
"Review PR #XXXXX" Review an existing PR with independent analysis
"Continue working on #XXXXX" Resume from where you left off

How It Works

The pipeline is orchestrated by Review-PR.ps1, which runs 4 steps:

Step 0: Branch Setup

Creates a clean review branch from main and squash-merges the PR onto it. If there are merge conflicts, the script posts a comment on the PR and exits.

Step 1: Gate

Runs verify-tests-fail.ps1 to confirm tests actually catch the bug:

  • Tests must fail without the fix (on the main baseline)
  • Tests must pass with the fix applied

If the PR has no tests, the gate can invoke write-ui-tests or write-xaml-tests to create them. The gate result is passed forward as context.

Step 2: PR Review (3-phase skill)

Invokes the pr-review skill via Copilot CLI, which runs three phases:

Pre-Flight — Reads the linked issue, PR description, comments, and classifies changed files. No code analysis yet — just context gathering.

Try-Fix — The core of the review. Four AI models explore independent fix ideas sequentially, each learning from prior attempts:

Order Model
1 Claude Opus 4.6
2 Claude Sonnet 4.6
3 GPT-5.3-Codex
4 Gemini-3-Pro-Preview

Each model:

  1. Generates an independent fix idea (without seeing the PR's solution first)
  2. Implements it
  3. Runs tests to verify

After all 4 attempts, a cross-pollination round asks each model if it has new ideas based on all results so far. This repeats until all models say "NO NEW IDEAS" (max 3 rounds).

The best fix is selected by comparing all passing candidates on simplicity, robustness, and codebase consistency.

Report — Writes the final review recommendation (APPROVE or REQUEST CHANGES) with a comparison of all fix candidates.

Step 3: Post AI Summary

Posts the review as a collapsible comment on the PR, including the gate result, try-fix comparison table, and final recommendation.

Step 4: Apply Labels

Parses the phase output files and applies s/agent-* labels to the PR:

Label Meaning
s/agent-reviewed PR was reviewed by agent (always applied)
s/agent-approved Agent recommends approval
s/agent-changes-requested Agent recommends changes
s/agent-review-incomplete Agent couldn't complete all phases
s/agent-gate-passed Tests verified to catch the bug
s/agent-gate-failed Could not verify tests catch the bug
s/agent-fix-win Agent found a better fix than the PR
s/agent-fix-pr-picked PR's fix was the best option
s/agent-fix-implemented (Manual) Maintainer confirms author adopted agent's suggestion

Skills Used

Skill Purpose
pr-review Orchestrates the 3-phase review (Pre-Flight, Try-Fix, Report)
code-review Deep code review with MAUI-specific rules (345 rules across 20 categories)
try-fix Single-model fix attempt — generates, implements, and tests one approach
verify-tests-fail-without-fix Confirms tests catch the bug (fail without fix, pass with fix)
write-ui-tests Creates UI tests if the PR has none
write-xaml-tests Creates XAML unit tests for parsing/compilation issues
evaluate-pr-tests Evaluates test quality, coverage, and appropriateness
pr-finalize Verifies PR title/description match implementation before merge

State Tracking

All phase output is written to CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/{phase}/content.md. This allows resuming interrupted sessions and tracking fix attempts.

When NOT to Use

Task Use Instead
Just run tests manually Sandbox Agent
Only write tests without fixing Write Tests Agent
Quick manual validation Sandbox Agent
Extract lessons from a completed PR Learn From PR Agent

File Locations

File Purpose
.github/scripts/Review-PR.ps1 Main orchestrator script
.github/skills/pr-review/SKILL.md 3-phase review skill definition
.github/skills/code-review/SKILL.md Deep code review rules
.github/pr-review/pr-preflight.md Pre-Flight phase instructions
.github/pr-review/pr-gate.md Gate phase instructions
.github/pr-review/pr-report.md Report phase instructions
.github/skills/try-fix/SKILL.md Fix exploration skill
.github/scripts/shared/Update-AgentLabels.ps1 Label management
.github/docs/agent-labels.md Label system documentation

Related Documentation

Clone this wiki locally