Agentic Development: LLMs Keeping Themselves in Check

Here's a flow I now use for agentic development:

Code in an agentic IDE → Submit PR → Greptile or CodeRabbit pick up the PR and perform code review → Implement fixes as suggested → Re-run AI code review → rinse-repeat → Merge in change

LLMs keeping themselves in check

I usually add the following to the top of the AI prompt that Greptile provides:

What this does is doesn't assume that the prompt provided should be fixed but first asks the agent on the other end whether or not the claim is valid or not. If not, the agent will say so. If so, the agent will agree and then make suggestions on the fix.

AI code review tools are great at catching issues, but they can also flag things that aren't actually problems. By adding this preamble, you're effectively having the LLM do a reasonableness check on its own output. It's a simple prompt that saves a lot of back-and-forth.