AI Code Reviews on GitHub

Introduction

You've probably heard of tools like Code Rabbit, What the Diff, and the newest Cursor's BugBot that perform AI-powered code reviews on pull requests, right? But have you ever wondered how they work under the hood? Well, I have, and in this article, I'll show you how to create a workflow to automatically review your PRs simply by adding a label to them.

What we'll create

We'll create a GitHub Action that:

Is triggered when an ai-review label is added to the PR
Reads the modified files (code "diff")
Sends the content to the Gemini API with a prompt requesting code review
Automatically comments the AI's result on the pull request

Prerequisites

Before we start, you'll need:

A GitHub repository with an open PR for testing
Access to Google AI Studio to generate a Gemini API key
Permissions to create secrets and configure workflows in your repository
A pair of scissors without a point (just kidding)

Setting up access and secrets

Getting the Gemini API key

Before configuring the secret on GitHub, you need to obtain your API key from Google AI Studio:

Go to Google AI Studio
Sign in with your Google account
Click "Get API key" in the top right corner
Select "Create API key" to generate a new key
Copy the generated key (it will only be displayed once)
Save this key in a secure location - you'll need it for the next step

Creating the `GEMINI_API_KEY` Secret on GitHub

Go to your repository on GitHub
Navigate to Settings > Secrets and variables > Actions
Click "New repository secret"
Name: GEMINI_API_KEY
Value: paste the API key you copied in the previous step

Creating the GitHub Action

Let's build the workflow step by step. First, create a new file in your repository:

.github/workflows/ai-review.yml

Step 1: Configuring the trigger

We start by defining when our action should run. We want it to run only when a specific label is added to the PR:

name: Review Pull Request with Gemini

on:
  pull_request:
    types: [labeled] # Only trigger when labels are added

jobs:
  review:
    runs-on: ubuntu-latest
    if: github.event.label.name == 'ai-review'

Explanation:

types: [labeled] - The action only runs when a label is added (not when the PR is created)
if: github.event.label.name == 'ai-review' - Only runs if the label is exactly ai-review

Step 2: Checking out the code

We need to download the PR code to analyze it:

steps:
  - name: Checkout PR code
    uses: actions/checkout@v4
    with:
      fetch-depth: 0

Explanation:

fetch-depth: 0 - Downloads the entire git history so we can compare with the base branch

Step 3: Identifying modified files

Now let's find out which files were changed in the PR:

- name: Get changed files
  id: files
  run: |
    # Fetch the base branch (usually main or master)
    git fetch origin ${{ github.event.pull_request.base.ref }}

    # Get changed files between base branch and current PR
    echo "CHANGED_FILES=$(git diff --name-only origin/${{ github.event.pull_request.base.ref }}...HEAD | grep '\.js\|\.ts\|\.tsx\|\.md\|\.css\|\.json\|\.yml\|\.yaml\|\.sh\|\.md' | tr '\n' ',' | sed 's/,$//')" >> $GITHUB_OUTPUT

Explanation:

git diff --name-only - Lists only the names of modified files
grep - Filters only code/text files (excluding images, binaries, etc.)
tr '\n' ',' - Converts line breaks to commas for easier processing

Step 4: Reading file contents

Let's read the content of all modified files:

- name: Read file contents
  id: read
  run: |
    mkdir -p output
    CHANGED_FILES="${{ steps.files.outputs.CHANGED_FILES }}"
    if [ -z "$CHANGED_FILES" ]; then
      echo "No changed files found" > output/combined.txt
    else
      for file in $(echo "$CHANGED_FILES" | tr ',' '\n'); do
        if [ -f "$file" ]; then
          echo ">>>>> $file" >> output/combined.txt
          cat "$file" >> output/combined.txt
          echo -e "\n\n" >> output/combined.txt
        fi
      done
    fi

Explanation:

We create a combined.txt file with all the modified code (the entire diff)
Each file is preceded by >>>>> filename for identification
If there are no modified files, we create an empty file

Step 5: Sending to Gemini API

Now the most interesting part - let's send the code to the AI for analysis:

- name: Send to Gemini API
  id: gemini
  if: steps.files.outputs.CHANGED_FILES != ''
  env:
    GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
  run: |
    jq -n \
      --arg prompt "You are an assistant that performs code reviews. Given the code below, comment on improvements, issues, or best practices that apply. Code:" \
      --rawfile code output/combined.txt \
      '{
        "contents": [{
          "parts": [{
            "text": ($prompt + "\n\n" + $code)
          }]
        }]
      }' > request.json

    # Make the API request using the JSON file
    RESPONSE=$(curl -s -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-pro:generateContent?key=${GEMINI_API_KEY}" \
      -H "Content-Type: application/json" \
      -d @request.json)

    # Extract the review text from response
    REVIEW_TEXT=$(echo "$RESPONSE" | jq -r '.candidates[0].content.parts[0].text // "Error processing API response"')

    echo "REVIEW<<EOF" >> $GITHUB_OUTPUT
    echo "$REVIEW_TEXT" >> $GITHUB_OUTPUT
    echo "EOF" >> $GITHUB_OUTPUT

Explanation:

jq -n - Creates a JSON with the prompt and code
jq -r - Extracts the text from the AI response
REVIEW<<EOF - Saves the result in a GitHub Actions output variable

Step 6: Commenting on the PR

Finally, let's post the analysis result on the pull request itself:

- name: Comment on PR
  if: steps.files.outputs.CHANGED_FILES != ''
  uses: peter-evans/create-or-update-comment@v4
  with:
    token: ${{ secrets.GITHUB_TOKEN }}
    issue-number: ${{ github.event.pull_request.number }}
    body: |
      🤖 **Gemini Code Review**
      ---
      ${{ steps.gemini.outputs.REVIEW }}

Explanation:

Uses the peter-evans/create-or-update-comment action to comment on the PR. Check out the documentation for more details.
${{ secrets.GITHUB_TOKEN }} - Automatic GitHub token for authentication
${{ github.event.pull_request.number }} - Current PR number

Complete file

Here's the complete file you should save as .github/workflows/ai-review.yml:

name: Review Pull Request with Gemini

on:
  pull_request:
    types: [labeled] # Only trigger when labels are added

jobs:
  review:
    runs-on: ubuntu-latest
    if: github.event.label.name == 'ai-review'

    steps:
      - name: Checkout PR code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Get changed files
        id: files
        run: |
          # Fetch the base branch (usually main or master)
          git fetch origin ${{ github.event.pull_request.base.ref }}

          # Get changed files between base branch and current PR
          echo "CHANGED_FILES=$(git diff --name-only origin/${{ github.event.pull_request.base.ref }}...HEAD | grep '\.js\|\.ts\|\.tsx\|\.md\|\.css\|\.json\|\.yml\|\.yaml\|\.sh\|\.md' | tr '\n' ',' | sed 's/,$//')" >> $GITHUB_OUTPUT

      - name: Read file contents
        id: read
        run: |
          mkdir -p output
          CHANGED_FILES="${{ steps.files.outputs.CHANGED_FILES }}"
          if [ -z "$CHANGED_FILES" ]; then
            echo "No changed files found" > output/combined.txt
          else
            for file in $(echo "$CHANGED_FILES" | tr ',' '\n'); do
              if [ -f "$file" ]; then
                echo ">>>>> $file" >> output/combined.txt
                cat "$file" >> output/combined.txt
                echo -e "\n\n" >> output/combined.txt
              fi
            done
          fi

      - name: Send to Gemini API
        id: gemini
        if: steps.files.outputs.CHANGED_FILES != ''
        env:
          GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
        run: |
          # Create the request JSON file to avoid argument list too long error
          jq -n \
            --arg prompt "You are an assistant that performs code reviews. Given the code below, comment on improvements, issues, or best practices that apply. Code:" \
            --rawfile code output/combined.txt \
            '{
              "contents": [{
                "parts": [{
                  "text": ($prompt + "\n\n" + $code)
                }]
              }]
            }' > request.json

          # Make the API request using the JSON file
          RESPONSE=$(curl -s -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-pro:generateContent?key=${GEMINI_API_KEY}" \
            -H "Content-Type: application/json" \
            -d @request.json)

          # Extract the review text from response
          REVIEW_TEXT=$(echo "$RESPONSE" | jq -r '.candidates[0].content.parts[0].text // "Error processing API response"')

          echo "REVIEW<<EOF" >> $GITHUB_OUTPUT
          echo "$REVIEW_TEXT" >> $GITHUB_OUTPUT
          echo "EOF" >> $GITHUB_OUTPUT

      - name: Comment on PR
        if: steps.files.outputs.CHANGED_FILES != ''
        uses: peter-evans/create-or-update-comment@v4
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          issue-number: ${{ github.event.pull_request.number }}
          body: |
            🤖 **Gemini Code Review**
            ---
            ${{ steps.gemini.outputs.REVIEW }}

Let's test it

Go to the GitHub labels editing page

https://github.com/YOUR-USERNAME/YOUR-PROJECT/issues/labels

Create the "ai-review" label

Creating the "ai-review" label on GitHub

Go to the PR on GitHub
Add the "ai-review" label to your PR

Adding the "ai-review" label to the PR

The Action will be triggered and, in a few moments, a comment with the AI review will appear on your PR

AI Code Reviews on GitHub

From there, whenever you need the review to be redone, just remove and re-add the label.

Tips and customizations

You can adjust the prompt sent to Gemini to focus on security, performance, or specific patterns
To avoid unnecessary calls, limit the files analyzed by extension (like only .js, .css, etc...)
Obviously the Action can be adapted to work with other AIs (Claude, GPT, Mistral...), just be careful with the credit card bill! ⚠️
It's very easy to apply the suggested fixes by copying the PR comment and pasting it into Cursor's Cascade, without needing to enable Max Mode as is the case with BugBot

Conclusion

It's possible to integrate AI to perform code reviews directly on your GitHub without committing to another subscription on your credit card. Of course, in this case, the responsibility of iterating and improving the solution also falls on you 😅.

What we learned

In this tutorial, you learned to:

Configure a GitHub Action that responds to specific labels
Identify modified files in pull requests
Integrate with the Gemini API for code analysis
Automate comments on PRs

Next steps

Now that you have the basics working, you can try:

Customizing the prompt to focus on specific aspects of your project
Adding filters for different file types
Integrating with other AIs like Claude or GPT
Creating specific workflows for different types of review

Useful resources

Remember: AI is a powerful tool, but it doesn't replace human review. Use it as a complement to identify potential problems and improvements, but always keep a critical eye on the code being reviewed.

Let's go! 🚀

Introduction

What we'll create

Prerequisites

Setting up access and secrets

Getting the Gemini API key

Creating the GEMINI_API_KEY Secret on GitHub

Creating the GitHub Action

Step 1: Configuring the trigger

Step 2: Checking out the code

Step 3: Identifying modified files

Step 4: Reading file contents

Step 5: Sending to Gemini API

Step 6: Commenting on the PR

Complete file

Let's test it

Tips and customizations

Conclusion

What we learned

Next steps

Useful resources

Creating the `GEMINI_API_KEY` Secret on GitHub