“We stuck with it for maybe a few weeks. And I remember we turned it off for a day or two, and we definitely got feedback that hey, I miss that. That was the clear test.”
VP of Engineering, Scale
Aakash Sabharwal
Q&A
At some point last year it was obvious that we're generating a lot more code faster, people want to go faster, and as always code review is going to be a bottleneck. It was always a bottleneck, and now even more so, and these coding agents are getting smarter.
We evaluated Greptile, and I think the initial feedback was that this is annoying. I get this score on my PR, or it’s noisy. And then we stuck with it for maybe a few weeks. And I remember we turned it off for a day or two, and we definitely got feedback that hey, I miss that. That was the clear test.
So we adopted it, we rolled it out everywhere.
The metrics I was looking for were, frankly, developer satisfaction, and would it actually help PR approval times and PR review times.
The thesis was that if Greptile does a first review on a PR and it reaches a good score, and then a human reviews it, the human also feels a little more comfortable reviewing it.
We have rules on promotion of simple diffs below a certain complexity based on a Greptile review — that can go to staging or auto-promote. So it's part of that kind of automation, or at least I believe we're getting there.
Right now it is pretty low. It's super simple diffs — if you're changing a fixture file or changing a log line, and it's deemed low risk and has a Greptile approval, it can get included in the next release. But that's a pretty small percentage of diffs.
The other way to solve it is if you're more sure of your production monitoring and testing, then you can rely on it more. We want to get to a place where we feel comfortable increasing that percentage.
I would put the impact as: we're definitely merging faster, shipping faster, particularly on smaller, less complex diffs.
And the second thing is that there's just more hygiene on the PR. It's easier for reviewers when they come in — there's more context. Sure, now Claude and Codex are also doing that. But Greptile creates a good experience for a reviewer and gives them confidence that it's at a high threshold of reliability.
It's been a great experience.
Scale lets a Greptile review promote code
As code volume climbed and review stayed the bottleneck, Scale rolled Greptile out to ~650 engineers working across Python, Go, TypeScript, and React.
Early feedback was that it was annoying and noisy. Scale kept it running a few weeks, turned it off, and engineers asked for it back. It started on one of the larger repos. Other organizations asked for it from there.
Greptile has caught 100K+ bugs. A human still reviews nearly all of them, and the argument for going first is that the human arrives at a PR that already has a score and context on it.
Simple diffs promote on a Greptile approval without one — a changed fixture file, a log line, anything under a complexity threshold and deemed low risk goes into the next release. That is a small percentage of diffs today. Aakash Sabharwal says what has to improve before it grows: production monitoring and testing the team trusts enough to lean on.
Learn more about Greptile
Greptile consistently catches more bugs in large repos
