# Observations From Building a PR Review Loop

DevFeed: [Observations From Building a PR Review Loop](<https://devfeed.tech/articles/observations-from-building-a-pr-review-loop-32358.md>)

Original publisher: [Read original article](<https://joshtronic.com/2026/08/02/observations-building-pr-review-loop/>)

Author: Josh Sherman

Published: 2026-08-02T00:00:00Z

Content type: opinion

Language: en

Sources: [Josh Sherman](<https://devfeed.tech/sources/josh-sherman.md>)

Topics: [Pull Request](<https://devfeed.tech/topics/pull-request.md>), [Code](<https://devfeed.tech/topics/code.md>), [Security](<https://devfeed.tech/topics/security.md>), [Git](<https://devfeed.tech/topics/git.md>), [forgejo](<https://devfeed.tech/topics/forgejo.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [changes](<https://devfeed.tech/tags/changes.md>), [code](<https://devfeed.tech/tags/code.md>), [git](<https://devfeed.tech/tags/git.md>), [review](<https://devfeed.tech/tags/review.md>), [security](<https://devfeed.tech/tags/security.md>)

## AI overview

The author describes building an agent-driven pull request review loop and the problems encountered when reviewing large changes. A diff-only reviewer repeatedly requested changes, contributing to review loops, while model changes were associated with a roughly halved approval rate. Security constraints intentionally prevent the reviewer from using command-line tools, so a separate worker agent verifies feedback against the working tree and attempts remediation.

## Source excerpt

Building my own agent has been fun. Sure, I could have opted for an off-the-shelf solution, but I wanted control. The ironic part is that I'm using that control to offload more control to the agent. Recent experiments have included letting the agent work on itself. Super meta, but didn't quite go as planned considering the guardrails I have in place around the size of PRs. Agent work tends to be heavy lifting, whereas most project work is small and fits nicely inside the constraints. Part of the issue there is that the PR reviewer would constantly request changes on those larger PRs. Changes would happen, PR message body would diverge. Infinite loop death spiral until eventually I was called. Around this same time, I was also tracking some issues that started to creep in after I changed some models from Opus 4.8 to Opus 5. Approval rate dropped by about half. I like to think that each new model is going to be better than the last, so I thought perhaps it was time to revisit the reviewer to see if there was something we were missing. The review mechanic By design, the review persona is pretty dumb. Not dumb in the sense that I give it a lesser model or anything. Dumb in that it just doesn't have the full picture of things. It gets the PR diff as a blob of text and that's it. No git worktree, no way to read files or grep around. No commit messages or recent changes to compare against. I don't review PRs this way. I click around and look at adjacent code. I'll git checkout the branch locally and poke around. I try to gather as much as I can to give feedback with more substance than nit: stray line break or the like. Because of this, a lot of the feedback from the reviewer was effectively LGTM, but I can't really confirm anything so... lol good luck. Then I'd get pinged to take a closer look. Safety first It's probably not fair to say the reviewer is dumb. The reviewer is gated by security checks. I'm the only user on my personal Forgejo instance, but that doesn't mean