# AI CODE REVIEW BENCHMARKS

An evaluation of 5 AI code review tools across 50 real-world bugs from production codebases (2025). See which tools actually catch the issues that matter.

- 5 tools
- 50 bugs
- 5 repos
- 2025

## Overview

We compare 5 AI code review tools on 50 real-world pull requests to surface practical differences in how they catch bugs, manage signal versus noise, support multiple languages, and impact review quality.

Each tool was evaluated with default settings (no custom rules or fine-tuning). We measured bug-catch rates, comment quality, noise levels, time to review, and setup experience to reflect how these tools perform in everyday use.

All PRs come from public, verifiable repositories, so you can inspect the sources and reproduce the runs on your own. If you'd like the exact protocol, see the Methodology section.

### Bug Detection by Severity Level

| Severity | Greptile | Bugbot | Copilot | CodeRabbit | Graphite |
| --- | ---: | ---: | ---: | ---: | ---: |
| Critical | 58% | 58% | 50% | 33% | 17% |
| High | 100% | 64% | 57% | 36% | 0% |
| Medium | 89% | 56% | 78% | 56% | 11% |
| Low | 87% | 53% | 87% | 53% | 0% |

## Methodology

The dataset covers 5 open-source GitHub repositories in different languages. From each, 10 real bug-fix PRs were traced back to the commits that introduced the bugs. Extremely large or single-file changes were excluded to keep the set realistic.

For each case, two branches were created: one before the bug and one after the fix. A fresh PR reintroduced the original change and was replicated across 5 clean forks, one per code review tool. Each tool had full repository access, including the PR diff and base branches.

All tools ran in their hosted cloud plans with default settings (no custom rules), and reviews were triggered by opening the PR or invoking the bot. A bug counted as "caught" only when the tool explicitly identified the faulty code in a line-level comment and explained the impact. All results were verified against the known bug.

Note that this evaluation was conducted in July 2025, and these tools evolve quickly, so performance may change over time. Scoring considered only detection of the original bug; false positives, style suggestions, and unrelated comments did not affect the catch rate.

### Test Dataset

| Language | Repository | Description |
| --- | --- | --- |
| Python | [Sentry](https://github.com/getsentry/sentry) | Error tracking & performance monitoring |
| TypeScript | [Cal.com](https://github.com/calcom/cal.com) | Open source scheduling infrastructure |
| Go | [Grafana](https://github.com/grafana/grafana) | Monitoring & observability platform |
| Java | [Keycloak](https://github.com/keycloak/keycloak) | Identity & access management |
| Ruby | [Discourse](https://github.com/discourse/discourse) | Community discussion platform |

## Bug Catch Performance

Greptile led with an 82% catch rate, 41% higher than Bugbot (58%). The rest stack clearly: Bugbot and Copilot in the mid-50s, CodeRabbit at 44%, and Graphite at 6%.

### Overall Performance

| Greptile | Bugbot | Copilot | CodeRabbit | Graphite |
| ---: | ---: | ---: | ---: | ---: |
| 82% | 58% | 54% | 44% | 6% |

### CRITICAL

| Greptile | Bugbot | Copilot | CodeRabbit | Graphite |
| ---: | ---: | ---: | ---: | ---: |
| 58% | 58% | 50% | 33% | 17% |

### HIGH

| Greptile | Bugbot | Copilot | CodeRabbit | Graphite |
| ---: | ---: | ---: | ---: | ---: |
| 100% | 64% | 57% | 36% | 0% |

### MEDIUM + LOW

| Greptile | Bugbot | Copilot | CodeRabbit | Graphite |
| ---: | ---: | ---: | ---: | ---: |
| 88% | 58% | 55% | 55% | 6% |

## Case Library

Performance varies by repository and language. The tables list every PR in the test set with a one-line bug summary, severity, and whether each tool caught it. Tool names link to the tool's run, and each ✓/✗ links to the exact PR so you can review comments, summaries, and outputs.

The right choice depends on priorities. Some tools produced richer summaries, some were faster, and some were quieter. Use the tables to inspect cases that match your stack and tolerance for noise.

Caught = an explicit line-level PR comment that points to the faulty code and explains the impact. Summary-only mentions do not count.

### Sentry

| PR / Bug Description | Severity | [Greptile](https://github.com/ai-code-review-evaluation/sentry-greptile/pulls) | [Copilot](https://github.com/ai-code-review-evaluation/sentry-copilot/pulls) | [CodeRabbit](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pulls) | [Bugbot](https://github.com/ai-code-review-evaluation/sentry-cursor/pulls) | [Graphite](https://github.com/ai-code-review-evaluation/sentry-graphite/pulls) |
| --- | --- | :---: | :---: | :---: | :---: | :---: |
| Enhanced Pagination Performance for High-Volume Audit Logs; Importing non-existent OptimizedCursorPaginator | High | [✓](https://github.com/ai-code-review-evaluation/sentry-greptile/pull/1) | [✗](https://github.com/ai-code-review-evaluation/sentry-copilot/pull/1) | [✗](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pull/1) | [✗](https://github.com/ai-code-review-evaluation/sentry-cursor/pull/1) | [✗](https://github.com/ai-code-review-evaluation/sentry-graphite/pull/1) |
| Optimize spans buffer insertion with eviction during insert; Negative offset cursor manipulation bypasses pagination boundaries | Critical | [✗](https://github.com/ai-code-review-evaluation/sentry-greptile/pull/2) | [✗](https://github.com/ai-code-review-evaluation/sentry-copilot/pull/10) | [✓](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pull/2) | [✓](https://github.com/ai-code-review-evaluation/sentry-cursor/pull/2) | [✗](https://github.com/ai-code-review-evaluation/sentry-graphite/pull/2) |
| Support upsampled error count with performance optimizations; sample_rate = 0.0 is falsy and skipped | Low | [✓](https://github.com/ai-code-review-evaluation/sentry-greptile/pull/3) | [✗](https://github.com/ai-code-review-evaluation/sentry-copilot/pull/2) | [✗](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pull/3) | [✗](https://github.com/ai-code-review-evaluation/sentry-cursor/pull/3) | [✗](https://github.com/ai-code-review-evaluation/sentry-graphite/pull/3) |
| GitHub OAuth Security Enhancement; Null reference if github_authenticated_user state is missing | Critical | [✗](https://github.com/ai-code-review-evaluation/sentry-greptile/pull/4) | [✓](https://github.com/ai-code-review-evaluation/sentry-copilot/pull/3) | [✗](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pull/4) | [✓](https://github.com/ai-code-review-evaluation/sentry-cursor/pull/4) | [✗](https://github.com/ai-code-review-evaluation/sentry-graphite/pull/4) |
| Replays Self-Serve Bulk Delete System; Breaking changes in error response format | Critical | [✓](https://github.com/ai-code-review-evaluation/sentry-greptile/pull/5) | [✗](https://github.com/ai-code-review-evaluation/sentry-copilot/pull/4) | [✗](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pull/5) | [✗](https://github.com/ai-code-review-evaluation/sentry-cursor/pull/5) | [✗](https://github.com/ai-code-review-evaluation/sentry-graphite/pull/5) |
| Span Buffer Multiprocess Enhancement with Health Monitoring; Inconsistent metric tagging with 'shard' and 'shards' | Medium | [✓](https://github.com/ai-code-review-evaluation/sentry-greptile/pull/6) | [✓](https://github.com/ai-code-review-evaluation/sentry-copilot/pull/5) | [✗](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pull/6) | [✗](https://github.com/ai-code-review-evaluation/sentry-cursor/pull/6) | [✗](https://github.com/ai-code-review-evaluation/sentry-graphite/pull/6) |
| Implement cross-system issue synchronization; Shared mutable default in dataclass timestamp | Medium | [✓](https://github.com/ai-code-review-evaluation/sentry-greptile/pull/7) | [✓](https://github.com/ai-code-review-evaluation/sentry-copilot/pull/6) | [✓](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pull/7) | [✓](https://github.com/ai-code-review-evaluation/sentry-cursor/pull/7) | [✗](https://github.com/ai-code-review-evaluation/sentry-graphite/pull/7) |
| Reorganize incident creation / issue occurrence logic; Using stale config variable instead of updated one | High | [✓](https://github.com/ai-code-review-evaluation/sentry-greptile/pull/8) | [✗](https://github.com/ai-code-review-evaluation/sentry-copilot/pull/7) | [✓](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pull/8) | [✗](https://github.com/ai-code-review-evaluation/sentry-cursor/pull/8) | [✗](https://github.com/ai-code-review-evaluation/sentry-graphite/pull/8) |
| Add ability to use queues to manage parallelism; Invalid queue.ShutDown exception handling | High | [✓](https://github.com/ai-code-review-evaluation/sentry-greptile/pull/9) | [✓](https://github.com/ai-code-review-evaluation/sentry-copilot/pull/8) | [✗](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pull/9) | [✗](https://github.com/ai-code-review-evaluation/sentry-cursor/pull/9) | [✗](https://github.com/ai-code-review-evaluation/sentry-graphite/pull/9) |
| Add hook for producing occurrences from the stateful detector; Incomplete implementation (only contains pass) | High | [✓](https://github.com/ai-code-review-evaluation/sentry-greptile/pull/10) | [✗](https://github.com/ai-code-review-evaluation/sentry-copilot/pull/9) | [✗](https://github.com/ai-code-review-evaluation/sentry-coderabbit/pull/10) | [✓](https://github.com/ai-code-review-evaluation/sentry-cursor/pull/10) | [✗](https://github.com/ai-code-review-evaluation/sentry-graphite/pull/10) |
| Total Catches |  | 8/10 | 4/10 | 3/10 | 4/10 | 0/10 |

### Cal.com

| PR / Bug Description | Severity | [Greptile](https://github.com/ai-code-review-evaluation/cal.com-greptile/pulls) | [Copilot](https://github.com/ai-code-review-evaluation/cal.com-copilot/pulls) | [CodeRabbit](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pulls) | [Bugbot](https://github.com/ai-code-review-evaluation/cal.com-cursor/pulls) | [Graphite](https://github.com/ai-code-review-evaluation/cal.com-graphite/pulls) |
| --- | --- | :---: | :---: | :---: | :---: | :---: |
| Async import of the appStore packages; Async callbacks in forEach creates unhandled promise rejections | Low | [✓](https://github.com/ai-code-review-evaluation/cal.com-greptile/pull/2) | [✓](https://github.com/ai-code-review-evaluation/cal.com-copilot/pull/2) | [✓](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pull/2) | [✓](https://github.com/ai-code-review-evaluation/cal.com-cursor/pull/2) | [✗](https://github.com/ai-code-review-evaluation/cal.com-graphite/pull/2) |
| feat: 2fa backup codes; Backup codes not invalidated after use | Critical | [✗](https://github.com/ai-code-review-evaluation/cal.com-greptile/pull/3) | [✗](https://github.com/ai-code-review-evaluation/cal.com-copilot/pull/3) | [✓](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pull/3) | [✗](https://github.com/ai-code-review-evaluation/cal.com-cursor/pull/3) | [✗](https://github.com/ai-code-review-evaluation/cal.com-graphite/pull/3) |
| fix: handle collective multiple host on destinationCalendar; Null reference error if array is empty | Medium | [✓](https://github.com/ai-code-review-evaluation/cal.com-greptile/pull/4) | [✓](https://github.com/ai-code-review-evaluation/cal.com-copilot/pull/4) | [✗](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pull/4) | [✓](https://github.com/ai-code-review-evaluation/cal.com-cursor/pull/4) | [✗](https://github.com/ai-code-review-evaluation/cal.com-graphite/pull/4) |
| feat: convert InsightsBookingService to use Prisma.sql raw queries; Potential SQL injection risk in raw SQL query construction | Critical | [✓](https://github.com/ai-code-review-evaluation/cal.com-greptile/pull/5) | [✓](https://github.com/ai-code-review-evaluation/cal.com-copilot/pull/5) | [✗](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pull/5) | [✗](https://github.com/ai-code-review-evaluation/cal.com-cursor/pull/5) | [✗](https://github.com/ai-code-review-evaluation/cal.com-graphite/pull/5) |
| Comprehensive workflow reminder management for booking lifecycle events; Missing database cleanup when immediateDelete is true | High | [✓](https://github.com/ai-code-review-evaluation/cal.com-greptile/pull/6) | [✗](https://github.com/ai-code-review-evaluation/cal.com-copilot/pull/6) | [✗](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pull/6) | [✓](https://github.com/ai-code-review-evaluation/cal.com-cursor/pull/6) | [✗](https://github.com/ai-code-review-evaluation/cal.com-graphite/pull/6) |
| Advanced date override handling and timezone compatibility improvements; Incorrect end time calculation using slotStartTime instead of slotEndTime | Medium | [✓](https://github.com/ai-code-review-evaluation/cal.com-greptile/pull/7) | [✓](https://github.com/ai-code-review-evaluation/cal.com-copilot/pull/7) | [✓](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pull/7) | [✗](https://github.com/ai-code-review-evaluation/cal.com-cursor/pull/7) | [✗](https://github.com/ai-code-review-evaluation/cal.com-graphite/pull/7) |
| OAuth credential sync and app integration enhancements; Timing attack vulnerability using direct string comparison | Critical | [✓](https://github.com/ai-code-review-evaluation/cal.com-greptile/pull/8) | [✗](https://github.com/ai-code-review-evaluation/cal.com-copilot/pull/8) | [✗](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pull/8) | [✗](https://github.com/ai-code-review-evaluation/cal.com-cursor/pull/8) | [✗](https://github.com/ai-code-review-evaluation/cal.com-graphite/pull/8) |
| SMS workflow reminder retry count tracking; OR condition causes deletion of all workflow reminders | High | [✗](https://github.com/ai-code-review-evaluation/cal.com-greptile/pull/9) | [✓](https://github.com/ai-code-review-evaluation/cal.com-copilot/pull/9) | [✓](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pull/9) | [✓](https://github.com/ai-code-review-evaluation/cal.com-cursor/pull/9) | [✗](https://github.com/ai-code-review-evaluation/cal.com-graphite/pull/9) |
| Add guest management functionality to existing bookings; Case sensitivity bypass in email blacklist | High | [✓](https://github.com/ai-code-review-evaluation/cal.com-greptile/pull/10) | [✓](https://github.com/ai-code-review-evaluation/cal.com-copilot/pull/10) | [✗](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pull/10) | [✗](https://github.com/ai-code-review-evaluation/cal.com-cursor/pull/10) | [✗](https://github.com/ai-code-review-evaluation/cal.com-graphite/pull/10) |
| feat: add calendar cache status and actions (#22532); Inaccurate cache status tracking due to unreliable updatedAt field | Low | [✓](https://github.com/ai-code-review-evaluation/cal.com-greptile/pull/11) | [✗](https://github.com/ai-code-review-evaluation/cal.com-copilot/pull/11) | [✗](https://github.com/ai-code-review-evaluation/cal.com-coderabbit/pull/11) | [✓](https://github.com/ai-code-review-evaluation/cal.com-cursor/pull/11) | [✗](https://github.com/ai-code-review-evaluation/cal.com-graphite/pull/11) |
| Total Catches |  | 8/10 | 6/10 | 4/10 | 5/10 | 0/10 |

### Grafana

| PR / Bug Description | Severity | [Greptile](https://github.com/ai-code-review-evaluation/grafana-greptile/pulls) | [Copilot](https://github.com/ai-code-review-evaluation/grafana-copilot/pulls) | [CodeRabbit](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pulls) | [Bugbot](https://github.com/ai-code-review-evaluation/grafana-cursor/pulls) | [Graphite](https://github.com/ai-code-review-evaluation/grafana-graphite/pulls) |
| --- | --- | :---: | :---: | :---: | :---: | :---: |
| Anonymous: Add configurable device limit; Race condition in CreateOrUpdateDevice method | High | [✓](https://github.com/ai-code-review-evaluation/grafana-greptile/pull/1) | [✗](https://github.com/ai-code-review-evaluation/grafana-copilot/pull/1) | [✗](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pull/1) | [✗](https://github.com/ai-code-review-evaluation/grafana-cursor/pull/1) | [✗](https://github.com/ai-code-review-evaluation/grafana-graphite/pull/1) |
| AuthZService: improve authz caching; Cache entries without expiration causing permanent permission denials | High | [✗](https://github.com/ai-code-review-evaluation/grafana-greptile/pull/2) | [✗](https://github.com/ai-code-review-evaluation/grafana-copilot/pull/2) | [✗](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pull/2) | [✓](https://github.com/ai-code-review-evaluation/grafana-cursor/pull/2) | [✗](https://github.com/ai-code-review-evaluation/grafana-graphite/pull/2) |
| Plugins: Chore: Renamed instrumentation middleware to metrics middleware; Undefined endpoint constants causing compilation errors | Critical | [✓](https://github.com/ai-code-review-evaluation/grafana-greptile/pull/3) | [✗](https://github.com/ai-code-review-evaluation/grafana-copilot/pull/3) | [✗](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pull/3) | [✗](https://github.com/ai-code-review-evaluation/grafana-cursor/pull/3) | [✗](https://github.com/ai-code-review-evaluation/grafana-graphite/pull/3) |
| Advanced Query Processing Architecture; Double interpolation risk | Critical | [✗](https://github.com/ai-code-review-evaluation/grafana-greptile/pull/4) | [✓](https://github.com/ai-code-review-evaluation/grafana-copilot/pull/4) | [✗](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pull/4) | [✓](https://github.com/ai-code-review-evaluation/grafana-cursor/pull/4) | [✗](https://github.com/ai-code-review-evaluation/grafana-graphite/pull/4) |
| Notification Rule Processing Engine; Missing key prop causing React rendering issues | Medium | [✓](https://github.com/ai-code-review-evaluation/grafana-greptile/pull/5) | [✗](https://github.com/ai-code-review-evaluation/grafana-copilot/pull/5) | [✓](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pull/5) | [✓](https://github.com/ai-code-review-evaluation/grafana-cursor/pull/5) | [✗](https://github.com/ai-code-review-evaluation/grafana-graphite/pull/5) |
| Dual Storage Architecture; Incorrect metrics recording methods causing misleading performance tracking | Medium | [✓](https://github.com/ai-code-review-evaluation/grafana-greptile/pull/6) | [✓](https://github.com/ai-code-review-evaluation/grafana-copilot/pull/6) | [✓](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pull/6) | [✓](https://github.com/ai-code-review-evaluation/grafana-cursor/pull/6) | [✓](https://github.com/ai-code-review-evaluation/grafana-graphite/pull/6) |
| Database Performance Optimizations; Incorrect error level logging | Low | [✓](https://github.com/ai-code-review-evaluation/grafana-greptile/pull/7) | [✓](https://github.com/ai-code-review-evaluation/grafana-copilot/pull/7) | [✓](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pull/7) | [✗](https://github.com/ai-code-review-evaluation/grafana-cursor/pull/7) | [✓](https://github.com/ai-code-review-evaluation/grafana-graphite/pull/7) |
| Frontend Asset Optimization; Deadlock potential during concurrent annotation deletion operations | High | [✓](https://github.com/ai-code-review-evaluation/grafana-greptile/pull/8) | [✓](https://github.com/ai-code-review-evaluation/grafana-copilot/pull/8) | [✓](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pull/8) | [✓](https://github.com/ai-code-review-evaluation/grafana-cursor/pull/8) | [✗](https://github.com/ai-code-review-evaluation/grafana-graphite/pull/8) |
| Advanced SQL Analytics Framework; enableSqlExpressions function always returns false, disabling SQL functionality | Critical | [✓](https://github.com/ai-code-review-evaluation/grafana-greptile/pull/9) | [✓](https://github.com/ai-code-review-evaluation/grafana-copilot/pull/9) | [✓](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pull/9) | [✓](https://github.com/ai-code-review-evaluation/grafana-cursor/pull/9) | [✓](https://github.com/ai-code-review-evaluation/grafana-graphite/pull/9) |
| Unified Storage Performance Optimizations; Race condition in cache locking | High | [✓](https://github.com/ai-code-review-evaluation/grafana-greptile/pull/10) | [✗](https://github.com/ai-code-review-evaluation/grafana-copilot/pull/10) | [✗](https://github.com/ai-code-review-evaluation/grafana-coderabbit/pull/10) | [✓](https://github.com/ai-code-review-evaluation/grafana-cursor/pull/10) | [✗](https://github.com/ai-code-review-evaluation/grafana-graphite/pull/10) |
| Total Catches |  | 8/10 | 5/10 | 5/10 | 7/10 | 3/10 |

### Keycloak

| PR / Bug Description | Severity | [Greptile](https://github.com/ai-code-review-evaluation/keycloak-greptile/pulls) | [Copilot](https://github.com/ai-code-review-evaluation/keycloak-copilot/pulls) | [CodeRabbit](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pulls) | [Bugbot](https://github.com/ai-code-review-evaluation/keycloak-cursor/pulls) | [Graphite](https://github.com/ai-code-review-evaluation/keycloak-graphite/pulls) |
| --- | --- | :---: | :---: | :---: | :---: | :---: |
| Fixing Re-authentication with passkeys; ConditionalPasskeysEnabled() called without UserModel parameter | Medium | [✓](https://github.com/ai-code-review-evaluation/keycloak-greptile/pull/1) | [✗](https://github.com/ai-code-review-evaluation/keycloak-copilot/pull/1) | [✗](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pull/1) | [✗](https://github.com/ai-code-review-evaluation/keycloak-cursor/pull/1) | [✗](https://github.com/ai-code-review-evaluation/keycloak-graphite/pull/1) |
| Add caching support for IdentityProviderStorageProvider .getForLogin operations; Recursive caching call using session instead of delegate | Critical | [✓](https://github.com/ai-code-review-evaluation/keycloak-greptile/pull/2) | [✗](https://github.com/ai-code-review-evaluation/keycloak-copilot/pull/2) | [✗](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pull/2) | [✗](https://github.com/ai-code-review-evaluation/keycloak-cursor/pull/2) | [✗](https://github.com/ai-code-review-evaluation/keycloak-graphite/pull/2) |
| Add AuthzClientCryptoProvider for authorization client cryptographic operations; Returns wrong provider (default keystore instead of BouncyCastle) | High | [✓](https://github.com/ai-code-review-evaluation/keycloak-greptile/pull/3) | [✗](https://github.com/ai-code-review-evaluation/keycloak-copilot/pull/3) | [✓](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pull/3) | [✗](https://github.com/ai-code-review-evaluation/keycloak-cursor/pull/3) | [✗](https://github.com/ai-code-review-evaluation/keycloak-graphite/pull/3) |
| Add rolling-updates feature flag and compatibility framework; Incorrect method call for exit codes | Medium | [✗](https://github.com/ai-code-review-evaluation/keycloak-greptile/pull/4) | [✗](https://github.com/ai-code-review-evaluation/keycloak-copilot/pull/4) | [✗](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pull/4) | [✗](https://github.com/ai-code-review-evaluation/keycloak-cursor/pull/4) | [✗](https://github.com/ai-code-review-evaluation/keycloak-graphite/pull/4) |
| Add Client resource type and scopes to authorization schema; Inconsistent feature flag bug causing orphaned permissions | High | [✗](https://github.com/ai-code-review-evaluation/keycloak-greptile/pull/5) | [✗](https://github.com/ai-code-review-evaluation/keycloak-copilot/pull/5) | [✗](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pull/5) | [✓](https://github.com/ai-code-review-evaluation/keycloak-cursor/pull/5) | [✗](https://github.com/ai-code-review-evaluation/keycloak-graphite/pull/5) |
| Add Groups resource type and scopes to authorization schema; Incorrect permission check in canManage() method | High | [✓](https://github.com/ai-code-review-evaluation/keycloak-greptile/pull/6) | [✓](https://github.com/ai-code-review-evaluation/keycloak-copilot/pull/6) | [✓](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pull/6) | [✓](https://github.com/ai-code-review-evaluation/keycloak-cursor/pull/6) | [✗](https://github.com/ai-code-review-evaluation/keycloak-graphite/pull/6) |
| Add HTML sanitizer for translated message resources; Lithuanian translation files contain Italian text | Low | [✓](https://github.com/ai-code-review-evaluation/keycloak-greptile/pull/7) | [✓](https://github.com/ai-code-review-evaluation/keycloak-copilot/pull/7) | [✓](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pull/7) | [✓](https://github.com/ai-code-review-evaluation/keycloak-cursor/pull/7) | [✗](https://github.com/ai-code-review-evaluation/keycloak-graphite/pull/7) |
| Implement access token context encoding framework; Wrong parameter in null check (grantType vs. rawTokenId) | Critical | [✓](https://github.com/ai-code-review-evaluation/keycloak-greptile/pull/8) | [✓](https://github.com/ai-code-review-evaluation/keycloak-copilot/pull/8) | [✓](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pull/8) | [✓](https://github.com/ai-code-review-evaluation/keycloak-cursor/pull/8) | [✗](https://github.com/ai-code-review-evaluation/keycloak-graphite/pull/8) |
| Implement recovery key support for user storage providers; Unsafe raw List deserialization without type safety | Medium | [✓](https://github.com/ai-code-review-evaluation/keycloak-greptile/pull/9) | [✗](https://github.com/ai-code-review-evaluation/keycloak-copilot/pull/9) | [✓](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pull/9) | [✓](https://github.com/ai-code-review-evaluation/keycloak-cursor/pull/9) | [✗](https://github.com/ai-code-review-evaluation/keycloak-graphite/pull/9) |
| Fix concurrent group access to prevent NullPointerException; Missing null check causing NullPointerException | Critical | [✓](https://github.com/ai-code-review-evaluation/keycloak-greptile/pull/10) | [✓](https://github.com/ai-code-review-evaluation/keycloak-copilot/pull/10) | [✗](https://github.com/ai-code-review-evaluation/keycloak-coderabbit/pull/10) | [✓](https://github.com/ai-code-review-evaluation/keycloak-cursor/pull/10) | [✗](https://github.com/ai-code-review-evaluation/keycloak-graphite/pull/10) |
| Total Catches |  | 8/10 | 4/10 | 5/10 | 6/10 | 0/10 |

### Discourse

| PR / Bug Description | Severity | [Greptile](https://github.com/ai-code-review-evaluation/discourse-greptile/pulls) | [Copilot](https://github.com/ai-code-review-evaluation/discourse-copilot/pulls) | [CodeRabbit](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pulls) | [Bugbot](https://github.com/ai-code-review-evaluation/discourse-cursor/pulls) | [Graphite](https://github.com/ai-code-review-evaluation/discourse-graphite/pulls) |
| --- | --- | :---: | :---: | :---: | :---: | :---: |
| FEATURE: automatically downsize large images; Method overwriting causing parameter mismatch | Medium | [✓](https://github.com/ai-code-review-evaluation/discourse-greptile/pull/1) | [✓](https://github.com/ai-code-review-evaluation/discourse-copilot/pull/1) | [✓](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pull/1) | [✓](https://github.com/ai-code-review-evaluation/discourse-cursor/pull/1) | [✗](https://github.com/ai-code-review-evaluation/discourse-graphite/pull/1) |
| FEATURE: per-topic unsubscribe option in emails; Nil reference non-existent TopicUser | High | [✓](https://github.com/ai-code-review-evaluation/discourse-greptile/pull/2) | [✓](https://github.com/ai-code-review-evaluation/discourse-copilot/pull/2) | [✓](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pull/2) | [✓](https://github.com/ai-code-review-evaluation/discourse-cursor/pull/2) | [✗](https://github.com/ai-code-review-evaluation/discourse-graphite/pull/2) |
| Add comprehensive email validation for blocked users; BlockedEmail.should_block? modifies DB during read | Critical | [✓](https://github.com/ai-code-review-evaluation/discourse-greptile/pull/3) | [✗](https://github.com/ai-code-review-evaluation/discourse-copilot/pull/3) | [✗](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pull/3) | [✓](https://github.com/ai-code-review-evaluation/discourse-cursor/pull/3) | [✗](https://github.com/ai-code-review-evaluation/discourse-graphite/pull/3) |
| Enhance embed URL handling and validation system; SSRF vulnerability using open(url) without validation | Critical | [✓](https://github.com/ai-code-review-evaluation/discourse-greptile/pull/4) | [✓](https://github.com/ai-code-review-evaluation/discourse-copilot/pull/4) | [✓](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pull/4) | [✓](https://github.com/ai-code-review-evaluation/discourse-cursor/pull/4) | [✗](https://github.com/ai-code-review-evaluation/discourse-graphite/pull/4) |
| Optimize header layout performance with flexbox mixins; Mixing float: left with flexbox causes layout issues | Low | [✗](https://github.com/ai-code-review-evaluation/discourse-greptile/pull/5) | [✗](https://github.com/ai-code-review-evaluation/discourse-copilot/pull/5) | [✓](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pull/5) | [✗](https://github.com/ai-code-review-evaluation/discourse-cursor/pull/5) | [✗](https://github.com/ai-code-review-evaluation/discourse-graphite/pull/5) |
| UX: show complete URL path if website domain is same as instance domain; String mutation with << operator | Medium | [✓](https://github.com/ai-code-review-evaluation/discourse-greptile/pull/6) | [✓](https://github.com/ai-code-review-evaluation/discourse-copilot/pull/6) | [✗](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pull/6) | [✓](https://github.com/ai-code-review-evaluation/discourse-cursor/pull/6) | [✗](https://github.com/ai-code-review-evaluation/discourse-graphite/pull/6) |
| scale-color $lightness must use $secondary for dark themes; Inconsistent theme color lightness affects visibility | Low | [✓](https://github.com/ai-code-review-evaluation/discourse-greptile/pull/7) | [✓](https://github.com/ai-code-review-evaluation/discourse-copilot/pull/7) | [✓](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pull/7) | [✓](https://github.com/ai-code-review-evaluation/discourse-cursor/pull/7) | [✗](https://github.com/ai-code-review-evaluation/discourse-graphite/pull/7) |
| FIX: proper handling of group memberships; Race conditions in async member loading | High | [✓](https://github.com/ai-code-review-evaluation/discourse-greptile/pull/8) | [✗](https://github.com/ai-code-review-evaluation/discourse-copilot/pull/8) | [✗](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pull/8) | [✗](https://github.com/ai-code-review-evaluation/discourse-cursor/pull/8) | [✗](https://github.com/ai-code-review-evaluation/discourse-graphite/pull/8) |
| FEATURE: Localization fallbacks (server-side); Thread-safety issue with lazy @loaded_locales | High | [✓](https://github.com/ai-code-review-evaluation/discourse-greptile/pull/9) | [✓](https://github.com/ai-code-review-evaluation/discourse-copilot/pull/9) | [✗](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pull/9) | [✗](https://github.com/ai-code-review-evaluation/discourse-cursor/pull/9) | [✗](https://github.com/ai-code-review-evaluation/discourse-graphite/pull/9) |
| FEATURE: Can edit category/host relationships for embedding; NoMethodError before_validation in EmbeddableHost | Critical | [✓](https://github.com/ai-code-review-evaluation/discourse-greptile/pull/10) | [✓](https://github.com/ai-code-review-evaluation/discourse-copilot/pull/10) | [✗](https://github.com/ai-code-review-evaluation/discourse-coderabbit/pull/10) | [✓](https://github.com/ai-code-review-evaluation/discourse-cursor/pull/10) | [✗](https://github.com/ai-code-review-evaluation/discourse-graphite/pull/10) |
| Total Catches |  | 9/10 | 7/10 | 5/10 | 7/10 | 0/10 |