Beyond LGTM: Bringing Speed and Quality to Code Reviews with AI Agents

At CosX we build complex backend infrastructure and AI-automated systems. Here's how we built an AI agent that reviews code like an expert engineer — and what happened next.

The Pain and the Promise of Code Reviews

Here at CosX, we make no compromises on shipping rock-solid code. As a team building complex backend infrastructure and AI-automated systems, we've always believed that code reviews are non-negotiable. They're our safety net, our mentorship loop, and our first line of defence against bad decisions slipping into production.

So when we rolled out the policy — "Every PR must be reviewed by at least two engineers before it touches main" — it felt like a victory for quality.

And it was...for a while.

Quality vs. Velocity: The Hidden Cost

As our team grew and deadlines got tighter, we started to feel the pressure.

• PRs would sit in review for hours, sometimes days blocking peer engineers.
• Reviewers were juggling unfamiliar code with other tasks.
• To keep things moving, we began to see 'LGTM 😊' style approvals that weren't really reviews.

We were slowing ourselves down — not because we didn't care, but because our code review process didn't scale with our swift velocity.

And ironically, quality was slipping, despite the guardrails we'd put in place.

Time to Innovate

We knew we needed a better way — something that could give us speed and quality.

So we gave ourselves a challenge: Could we build an AI agent that reviews code like an expert engineer?

Not a generic chatbot. Not a watered-down 'looks good' bot. But a real, helpful reviewer that runs on every PR.

How We Built the AI Review Agent

We used the OpenAI SDK and GitHub's Model Context Protocol (MCP) to build a multi-agents PR reviewer that lives alongside our repo and plays the role of an AI reviewer.

Every time a PR is created or updated, GitHub actions trigger a review app which further triggers our multi-agents PR reviewer. This then performs analysis on the PR diff and posts the review comments onto GitHub.

Designing Intelligent Agents

Creating intelligent agents is about designing the right prompts. A well-written prompt acts as the agent's blueprint for thinking, reasoning, and decision-making. It must clearly declare the agent's role, boundaries, goals, and expected behaviours. It is almost like writing a job description, a guide, and a mission statement all in one.

You are a senior software engineer.
Your job is to carefully review the given PR diff and flag
issues, errors, or incorrect control flow including:
- Logic errors or incorrect control flow
- Missing error handling
- Misuse of language features
- Data loss/misuse or performance issues

Your output should include:
- A clear explanation of the issue
- A code suggestion wherever possible
- A clear explanation of the bug found in this file

But writing the prompt is not a one-time effort. It's an iterative process. You start with a base design, observe how the agent responds, and refine the prompt based on gaps, misunderstandings, or new possibilities you discover. Each iteration helps sharpen the agent's intelligence, making it more aligned, reliable, and capable over time. This iterative approach helped us create agents that do not give spammy reviews!

Enhancing Team Communication: Google Chat Notifications for PRs

While the AI agent could instantly analyse and provide feedback on a PR, we didn't want to sideline the importance of human reviews. After all, code reviews are more than just finding bugs — they're a crucial part of the learning process. To keep everyone in the loop, and the 'why' moments that come from code reading are key to improving developer skills.

To keep everyone in the loop, we added a GitHub Action that sends a notification to our Google Chat whenever a PR is created or updated. This way, everyone always knows to jump in and contribute, whether it's right after the AI's review or during their own detailed review process.

- name: PR Created
  run: |
    curl -X POST '${{ secrets.GOOGLE_CHAT_WEBHOOK_URL }}' \
      -H 'Content-Type: application/json' \
      -d '{
        "text": "🔔 New PR opened\n\n*Owner:* ${{ github.repository_owner }}\n*Reviewer:* ${{ github.actor }}\n*Repository:* ${{ github.repository }}\n*PR Title:* ${{ github.event.pull_request.title }}\n*PR URL:* ${{ github.event.pull_request.html_url }}"
      }'

Now the whole squad sees it, can jump in early, or wait for the AI to surface feedback first. Simple. Effective. Kind of fun.

The Impact: Faster Reviews, Better Feedback

Within a few days of going live, the results were obvious:

• First-pass feedback landed in seconds instead of hours.
• Reviewers had a smaller starting point for their own review. PR Review coverage went up by nearly 80%.
• The average time-to-merge dropped by 70%, pulling down PR cycles by nearly 1.5 days.
• Estimated 20+ human-hours saved per week across the team, freeing engineers to focus more on building rather than closing reviews.

What's Next

With multiple iterations, adding code context and incorporating manual feedback we have been able to create a terrific agent. The obvious scope for improvement is to move past generating insights to creating actionable suggestions — the agent will simply fix 🔧 it on each suggestion, and we'll turn those notes into regular 'tune-ups' so the agent learns exactly what works (and what doesn't) in our codebase.

Final Thoughts

At CosX, our aim is to become the most autonomous AI Agency, revolutionising how we approach complex systems. As part of this mission, we've begun to sharpen the agents intelligence, making it more robust, but transforming entire workflows.

At our review agent, we like to call it 'Galactic Guardian' — one of the first agents we built for our team. A testament to the power of AI in improving developer workflows. Its ability to seamlessly integrate into our existing workflow has helped accelerate reviews, reduced review fatigue, and drastically sped up our time to merge.

But this is just the beginning: Galactic Guardian is only one piece of our larger plan. Each step we take in building and refining these agents brings us closer to our goal of being the first truly autonomous AI Agency — one where technology empowers our team to achieve more, faster and with greater precision.

For any technical details or deeper dives, feel free to reach out to us directly at admin@cosx.ai or stay tuned for upcoming blog posts.