Cover illustration for “GitHub Copilot Workspace for Pull Request Automation”

Essay Agentic IDE Reviews

GitHub Copilot Workspace for Pull Request Automation

Copilot can now automate an issue through to merge approval.

Riku Salminen8 min read

Agent Benchmarks · Agentic IDE Reviews

A blog and newsletter on Agent Bake-off.

Cover illustration for “Evaluating Agent Self-Correction and Recovery Behavior”

Essay Agent Benchmarks

Evaluating Agent Self-Correction and Recovery Behavior

Recovery-specific benchmarks reorder model rankings that pass/fail scores hide.

Priya Nandakumar · 8 min read

Cover illustration for “Cursor Agent Mode on Large Monorepos”

Essay Agentic IDE Reviews

Cursor Agent Mode on Large Monorepos

Scoped rules, tight ignores, and isolation keep agents useful.

Tomás Červenka · 8 min read

Cover illustration for “SWE-Bench Verified Results Interpreted for Practitioners”

Essay Agent Benchmarks

SWE-Bench Verified Results Interpreted for Practitioners

Understanding what SWE-bench Verified scores actually measure.

Tomás Červenka · 7 min read

GitHub Docs lists several clients from which developers can start the cloud agent, including the browser, IDEs, mobile, and CLI.
From GitHub Copilot Workspace for Pull Request Automation
Cover illustration for “Reproducibility Challenges in Coding Agent Benchmarks”

Essay Agent Benchmarks

Reproducibility Challenges in Coding Agent Benchmarks

Four interconnected problems make coding agent benchmark scores unreliable.

Deja Whitmore · 8 min read

Cover illustration for “Blind Scoring Panels for Agent Output Evaluation”

Essay Agent Benchmarks

Blind Scoring Panels for Agent Output Evaluation

Blind panels reveal how agents reach correct answers, not just whether they do.

Riku Salminen · 8 min read

Cover illustration for “Designing Task Suites That Expose Agent Weaknesses”

Essay Agent Benchmarks

Designing Task Suites That Expose Agent Weaknesses

Deliberate adversarial design catches agent failures that pass/fail benchmarks systematically miss.

Riku Salminen · 10 min read

Cover illustration for “Accuracy Metrics for Agentic Code Generation”

Essay Agent Benchmarks

Accuracy Metrics for Agentic Code Generation

Resolved rate conflates model capability with harness design and test quality in one number.

Riku Salminen · 9 min read

From the archive

  1. 01Essay Agent BenchmarksCost per Task Comparison Across Coding AgentsPriya Nandakumar · 9 min read
  2. 02Essay Agent BenchmarksSpeed Benchmarking Methodology for Coding AgentsRiku Salminen · 11 min read
  3. 03Essay Agentic IDE ReviewsVS Code GitHub Copilot Edits vs Cursor Composer Head to HeadPriya Nandakumar · 11 min read