Automating Code Reviews with AI: How Halodoc Scaled Quality Across the Full Stack

At Halodoc, code review is one of those practices everyone agrees is essential — but as AI adoption accelerates across our engineering organization, the growing volume of code changes has made it increasingly difficult for manual reviews alone to deliver consistent and timely feedback. The question is no longer whether AI belongs in the review process, but how to make it reliable enough to trust.

We maintain a Jenkins Global Library pipeline that serves as the foundation for CI/CD across our engineering organization. Each repository is mapped to its corresponding pipeline implementation, and the Global Library orchestrates build, test, security, code review, and deployment workflows based on repository type. This centralized approach enables consistent engineering practices across all repositories while allowing teams to focus on application development rather than pipeline maintenance.

With increasing release frequency and a growing volume of Merge Requests across engineering repositories, reviewers were repeatedly encountering the same categories of issues — missing ownership validation, unbounded database queries, insecure storage practices, and stack-specific implementation mistakes. A reviewer experienced in Java backend services might miss a concurrency issue in Swift, while those handling multiple Merge Requests simultaneously could overlook security-sensitive changes under workload pressure. In practice, review quality often depended more on reviewer context and availability than on the code itself.

To address this, we embedded an AI-powered code reviewer into the Global Library pipeline — triggered automatically whenever a Merge Request is created or updated. Every commit pushed to an active Merge Request is analyzed, with the system reviewing only the changed code, posting inline feedback directly on GitLab, and continuously updating or resolving comments as developers iterate. As a result, initial feedback time dropped from 30–45 minutes to just 1–2 minutes per Merge Request.

This post walks through how we built the system, the architectural decisions behind it, and the key learnings along the way.

Prerequisites for Adopting AI Code Review in Production

Before we could build anything, we first had to define what "good" looked like. The goal wasn't simply to generate AI comments on Merge Requests—it was to build an engineering capability that teams could trust as part of their everyday development workflow. That meant the solution needed to behave consistently, integrate seamlessly into existing CI/CD pipelines, and require minimal effort from development teams.

These became our non-negotiable design principles.

Execution Control

• Trigger AI code review automatically whenever a Merge Request pipeline is created or updated.
• Analyze every commit pushed to an active Merge Request.
• Apply repository-specific and target branch execution rules.
• Run as a mandatory stage in the CI/CD pipeline without requiring manual intervention.

Review Quality

• Post findings inline on the exact file and line where the issue exists.
• Analyze only the changed lines instead of the entire codebase.
• Produce consistent results for the same set of code changes.
• Generate feedback only when a concrete, actionable issue is identified.

Comment Lifecycle

• Update existing findings when an issue persists across subsequent commits.
• Avoid creating duplicate discussions for the same issue on every push.
• Automatically resolve comments once the underlying issue has been fixed.
• Prevent Merge Requests from being merged until all AI-generated findings have been addressed.

Cross-Stack Consistency

• Work out of the box across backend, web, iOS, Android, SRE, Data Engineering, and other engineering repositories
• Require no repository-specific setup from individual teams.
• Respect each repository's existing branching strategy and review policies.

Security and Credentials

The platform integrates with the organization’s existing secret management solution for AI credentials, ensuring that no credentials are embedded within repository code.

All repositories — including application, SRE, Data Engineering, and Security — are processed uniformly by the pipeline, with no exclusion list based on data sensitivity. Since code diffs are sent to external LLM providers (Gemini, Claude via AWS Bedrock) for analysis, teams adopting a similar approach should explicitly validate data handling, retention, and usage policies with their provider agreements rather than assuming default safeguards.

In addition, the review prompts apply a dedicated security checklist directly to the code under review. This includes detection of access control and authorization gaps, injection vulnerabilities, exposure of sensitive data such as tokens or PII in logs, missing input validation, insecure configurations (e.g., CORS misconfigurations or hardcoded credentials), and absence of rate limiting on sensitive endpoints. These checks help ensure issues are identified before reaching production, independent of how the underlying model providers handle input data.

These requirements directly shaped the architecture and influenced every design decision that followed, ensuring the solution could scale consistently across the engineering organisation while remaining reliable, secure, and easy to adopt.

Solution Architecture

After evaluating different approaches, we landed on a design that combines two AI models — Gemini CLI for fast diff-based reviews and Claude Sonnet through AWS Bedrock for deeper contextual analysis — integrated into the Jenkins Global Library as a dedicated review stage.

The flow is straightforward. When a Merge Request is created or updated, the Jenkins pipeline is triggered via a GitLab webhook. The diff and Merge Request context are fetched, a stack-specific prompt is selected, and the AI model is invoked. Findings are returned and posted as inline GitLab discussions. As developers push new changes, stale comments are automatically resolved.

The architecture has three main components:

• The Pipeline Stage — entry point that orchestrates the review flow
• The Prompt Templates — stack-specific instructions that guide the AI model
• The Comment Lifecycle Manager — keeps review discussions accurate as the Merge Request evolves

Component 1: The Pipeline Stage

The Pipeline Stage is the entry point for the entire review flow. Rather than asking teams to install plugins or maintain repository-specific review jobs, we embedded the review capability directly into the Jenkins Global Library, making it available automatically across all supported engineering repositories.

The review stage is triggered whenever a Merge Request is created or updated. Every commit pushed to an active Merge Request automatically initiates an AI code review. During execution, the pipeline gathers the required review context—including the source and target branches, the Git diff, and Merge Request metadata—and forwards it to the prompt engine. Repository-specific and target branch execution rules determine when the review stage is executed, ensuring consistent enforcement without requiring any manual intervention from developers.

To keep reviews focused and cost-efficient, only the changed source code is analyzed. Documentation and other non-code files (such as README.md and .txt files), along with code outside the diff, are excluded, reducing noise and ensuring that findings remain directly relevant to the developer's changes.

Component 2: The Prompt Templates

Wiring an AI model into the pipeline was the easy part. Getting it to produce useful output took more work.

Our first runs with generic prompts were noisy — the model commented on coding style, logging practices, and business logic decisions that were either acceptable in context or irrelevant to the specific change. To fix this, we built dedicated prompts for each repository category.

Each prompt defines:

• What to review — bugs and logic errors, anti-patterns, performance issues, security risks, test coverage gaps, and design flaws
• What to ignore — test files, business logic commentary, style preferences, and code outside the diff
• How to structure findings — concrete, actionable, and tied to specific lines

Each technology stack has its own prompt tailored to its specific failure modes:

Security reviews follow a predefined validation framework rather than a generic instruction to "look for vulnerabilities." The prompts evaluate a fixed set of controls covering authorization checks, query safety, credential exposure, rate limiting, and API lifecycle management. This made security findings far more consistent across repeated executions.

Repositories can also provide a AGENTS.md file with project-specific conventions, which the model uses to reduce false positives and align findings with team standards

Component 3: The Comment Lifecycle Manager

Generating review comments was only part of the problem. Without lifecycle management, every pipeline run would create a fresh set of discussions — quickly overwhelming developers with duplicate or outdated findings.

The Comment Lifecycle Manager synchronizes AI findings with existing GitLab discussions on every run:

• Existing finding — the discussion is updated, not duplicated
• New finding — a fresh inline discussion is created on the relevant file and line
• Resolved finding — if an issue no longer appears in the latest review, the corresponding discussion is automatically closed

Treating findings as long-lived review artifacts — rather than one-time messages — kept review threads accurate throughout the lifetime of every Merge Request.

Here's a real finding from the pipeline on one of our backend services. A change added a secret file path directly into a shell command:

Within minutes of the push, the AI reviewer flagged it as critical, and the discussion was automatically resolved once the fix landed — no manual follow-up required.

Model Selection and Trade-offs

Rather than relying on a single model, the platform dynamically routes reviews between two LLMs based on repository characteristics and change complexity. The selection is guided by the underlying requirement — whether prioritising speed and cost efficiency or deeper contextual understanding and accuracy.

By abstracting model invocation behind the Jenkins Global Library, the review framework remains independent of any single provider, enabling flexible selection of the optimal model per use case. This design allows us to balance speed, cost, and depth of analysis depending on repository needs.

Importantly, these trade-offs are based on observed production behavior rather than controlled benchmarking, reflecting practical engineering outcomes rather than formal model comparisons

Cost-Benefit Analysis

Building an AI-assisted code review platform also required evaluating its operational cost at scale. While large language models provide fast and consistent feedback, ensuring that the solution remained economically viable was an important design consideration.

Based on observed production usage, a typical large Merge Request review consumes approximately $0.17 in inference cost, while smaller Merge Requests incur significantly lower costs. Since implementation, the platform has processed 58,811 AI review invocations across engineering repositories — averaging approximately 9,800 reviews per month. At $0.17 per review, this translates to an estimated monthly inference spend in the range of $1,600–$1,700, depending on the mix of large versus small Merge Requests and variation in token consumption per review.

When weighed against the reduction in manual review effort, faster developer feedback, and consistent first-pass code analysis across engineering repositories, the operational cost remains relatively low. This makes AI-assisted code review a cost-effective addition to the CI/CD pipeline at this scale.

Lessons Learned

Building a production-ready AI Code Review platform involved much more than integrating a large language model into our CI/CD pipeline. Along the way, several engineering lessons shaped the final solution:

• Handling Large Merge Requests mattered. Exceptionally large Merge Requests occasionally triggered an "Argument list too long" error because the generated Git diff exceeded operating system command-line limits. Optimizing how review context was generated and passed to the LLM ensured reliable execution at scale.
• Prompt engineering mattered more than model selection. The biggest improvement in review quality came from narrowing each prompt to a small set of stack-specific failure modes—not from switching providers.
• Guardrails are as important as instructions. Telling the AI what to ignore was just as valuable as telling it what to look for. False positives dropped significantly once prompts became opinionated about scope.
• Lifecycle management is non-negotiable. Without it, review threads accumulate noise and developers stop trusting the tool. Automatically updating existing findings, resolving fixed issues, and preventing duplicate comments were essential to maintaining developer confidence.
• Treat it as an engineering system, not an AI experiment. Model choice matters, but rollout strategy, repository context, stack-specific prompts, operational reliability, and comment lifecycle management matter more than the choice of LLM alone.

Conclusion

Embedding AI code review into our Jenkins Global Library has transformed how engineering teams approach code quality. Developers now receive actionable, inline feedback on the exact lines that matter within 1–2 minutes of pushing a change — feedback that previously took 30–45 minutes to arrive through manual review. This automated first pass catches routine issues before a human reviewer even opens the Merge Request, freeing up reviewer time to focus on higher-value considerations such as architecture, business logic, and design decisions rather than repetitive checks.

Over the last six months, the platform has processed 58,811 AI review invocations across our engineering repositories, averaging approximately 9,800 reviews per month. Of these, 85% of findings were accepted directly, meaning the flagged issues were addressed in a follow-up commit before the Merge Request was merged, with no reviewer pushback. The remaining 15% required manual intervention, where a reviewer either overrode the finding or resolved it through discussion.

Rather than replacing human reviewers, AI has become the first line of defense for repetitive code quality checks, allowing engineers to focus on architecture, business logic, security considerations, and design decisions. By integrating AI directly into our CI/CD pipeline and supporting multiple LLM providers through a provider-agnostic architecture, we built a scalable code review framework that delivers consistent, high-quality feedback across all engineering repositories.

The journey reinforced an important principle: successful AI adoption is not defined by the choice of language model alone. Long-term success comes from combining reliable engineering practices, thoughtful prompt design, seamless CI/CD integration, and continuous iteration to build a solution that developers trust and use every day.

Join us

Scalability, reliability, and maintainability are the three pillars that govern what we build at Halodoc Tech. We are actively looking for engineers at all levels, and if solving hard problems with challenging requirements is your forte, please reach out to us with your resume at careers.india@halodoc.com.

About Halodoc

Halodoc is the number one all-around healthcare application in Indonesia. Our mission is to simplify and deliver quality healthcare across Indonesia, from Sabang to Merauke.

Since 2016, Halodoc has been improving health literacy in Indonesia by providing user-friendly healthcare communication, education, and information (KIE). In parallel, our ecosystem has expanded to offer a range of services that facilitate convenient access to healthcare, starting with Homecare by Halodoc as a preventive care feature that allows users to conduct health tests privately and securely from the comfort of their homes; My Insurance, which allows users to access the benefits of cashless outpatient services in a more seamless way; Chat with Doctor, which allows users to consult with over 20,000 licensed physicians via chat, video or voice call; and Health Store features that allow users to purchase medicines, supplements and various health products from our network of over 4,900 trusted partner pharmacies. To deliver holistic health solutions in a fully digital way, Halodoc offers Digital Clinic services including Haloskin, a trusted dermatology care platform guided by experienced dermatologists.

We are proud to be trusted by global and regional investors, including the Bill & Melinda Gates Foundation, Singtel, UOB Ventures, Allianz, GoJek, Astra, Temasek, and many more. With over USD 100 million raised to date, including our recent Series D, our team is committed to building the best personalized healthcare solutions — and we remain steadfast in our journey to simplify healthcare for all Indonesians.