Run Your First Monkey Test in 60 Seconds with the Free MonkeyTest CLI

MonkeyTest Core is a free, open-source AI agent that opens a real browser, works out what a user would try on your site, tries it, and reports what broke. This is the shortest path from nothing to a bug report.

Everything here uses the open-source CLI, which is AGPL-3.0 and has no paid tier, no seat limit and no run cap. The only thing you pay for is tokens from your own LLM provider.

Prerequisites

• Node 20 or newer. Check with node -v.
• An LLM API key. Gemini, OpenAI, Anthropic, Mistral, Groq, or any OpenAI-compatible endpoint including Ollama, vLLM and OpenRouter.
• A URL. Anything publicly reachable — production, staging, a Vercel preview, or localhost through a tunnel.

Step 1 — Install

npm install -g @mtai/monkeytest-core
npx playwright install chromium

The second line downloads the browser the agent drives. It is a real Chromium, not a headless emulation, because the bugs worth finding are the ones that only appear in a real rendering engine.

Step 2 — Give it a key

export GEMINI_API_KEY=...

Or OPENAI_API_KEY, ANTHROPIC_API_KEY, MISTRAL_API_KEY, GROQ_API_KEY — the CLI picks up whichever it finds.

A cheap, fast model is fine to start. Gemini Flash and GPT-4o-mini both handle exploration and execution well; you can move to a stronger model for planning later once you know it is worth it.

Step 3 — Run it

monkeytest run https://your-site.com

That single command does four things in sequence:

1. Explore. Crawls a handful of pages with a real browser and builds a map of what exists.
2. Plan. Asks the model what a real user would try here, and writes those flows out as a structured plan.
3. Run. Executes each flow step by step, capturing a screenshot per step plus console errors and failed network requests.
4. Triage. Groups what went wrong into deduplicated bug reports with severities and stable fingerprints.

Output looks roughly like this:

explore   crawled 8 pages
plan      4 flows a real user would try
run       3/4 flows passed
bug       [blocking] Checkout never submits
          observed: button click, no network call
          evidence: 3 screenshots

The process exits non-zero if it found a blocking bug, which is what makes it usable as a CI gate.

Step 4 — Read the output

Everything lands in a .monkeytest/ directory next to where you ran the command.

• plan.json — the flows the model decided to test, in plain structured JSON. Read it. If it has misunderstood what your site does, that is visible here rather than buried in a black box, and you can edit it by hand.
• shots/ — a screenshot per step. When a bug report says a button did nothing, the screenshot before and after is the proof.
• The run record — flows attempted, which passed, and the triaged bugs with their fingerprints.

Commit plan.json to git. That one habit is what turns this from a novelty into a test suite: the plan becomes a reviewable artefact your team can read, argue about, and diff in a pull request.

Step 5 — Fix something, then prove it

This is the part most AI testing tools skip.

# you ship a fix...
monkeytest rerun

rerun re-executes the exact plan from last time. No re-planning, which means no LLM cost and — more importantly — no plan drift. Yesterday’s run and today’s run tested the same things, so comparing them means something.

monkeytest diff prev latest

2 fixed, 0 new, 1 still broken, 0 regressed (net -2)

Because bugs carry stable fingerprints across runs, the diff can tell the difference between “this is a new bug”, “this is the same bug you already knew about”, and “this one came back”. That is the whole loop: find, fix, re-run, diff.

Step 6 — Put it in CI

The reason the CLI exits non-zero on a blocking bug is so it can gate a pull request like any other check. A minimal GitHub Actions workflow:

name: monkeytest

on:
  pull_request:

jobs:
  monkeytest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - run: npm install -g @mtai/monkeytest-core
      - run: npx playwright install --with-deps chromium

      - name: Run monkey test against the preview
        env:
          GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
        run: monkeytest run "${{ env.PREVIEW_URL }}"

      - name: Keep the evidence
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: monkeytest-evidence
          path: .monkeytest/

Two notes from running this in anger:

Use if: always() on the artifact upload. The runs you most want screenshots from are the ones that failed, and without always() the upload is skipped exactly then.

Point it at a preview deployment, not production. Vercel and Netlify both expose the preview URL to the workflow, so every pull request gets exercised by an agent against the code in that branch. That is where this earns its keep — bugs caught before review rather than after release.

For a deeper CI walkthrough, the docs have a dedicated recipe.

Using it as a library

The CLI is a thin wrapper. If you are embedding this in your own product or internal tooling, the same primitives are exported:

import { Monkey, autoLLMConfig, savePlan, loadPlan, saveRun, diffRuns } from '@mtai/monkeytest-core';

const monkey = new Monkey({ llm: autoLLMConfig({ quality: 'balanced' }) });

// Day 1 — discover
const { plan, run: a } = await monkey.executeAndTriage('https://example.com', {
  explore: { maxPages: 8, screenshotsDir: '.monkeytest/shots' },
  plan:    { maxFlows: 4 },
  run:     { screenshotsDir: '.monkeytest/shots' },
});
savePlan(plan, { dir: '.monkeytest' });
saveRun(a, { dir: '.monkeytest' });

// Day 2 — verify the fix
const stored = loadPlan('.monkeytest');
const { run: b } = await monkey.runFromPlan(stored.plan);
saveRun(b, { dir: '.monkeytest' });

console.log(diffRuns(a, b).stats);

You can also subscribe to a typed EventBus and pipe explorer, planner, runner and triager events into your own dashboard as the run happens.

Controlling cost

Three levers, in order of impact:

Mix models per phase. Exploration is high-volume and low-judgement; planning is the opposite. A cheap model exploring and a strong model planning gets you most of the quality at a fraction of the spend.

const monkey = new Monkey({
  llm: {
    explorer: { provider: 'gemini',    model: 'gemini-flash-lite-latest' },
    planner:  { provider: 'anthropic', model: 'claude-sonnet-4-5' },
    runner:   { provider: 'groq',      model: 'llama-3.3-70b-versatile' },
  },
});

Cap the crawl. maxPages and maxFlows bound how much work a run can do. Eight pages and four flows is plenty for a marketing site.

Rerun instead of run. rerun skips planning entirely, so day-to-day regression checks cost close to nothing. Reserve a full run for when the site has actually changed shape.

For a small site on a cheap model, a full run typically lands in the low single-digit cents.

When it goes wrong

Nothing gets past the login page. The agent needs credentials. Supply a test account through the auth configuration in the docs; without it, it can only see what a logged-out visitor sees.

The plan misunderstands the site. Read plan.json — this is exactly why it is a readable file. Edit it directly and rerun, or give planning a stronger model.

Bugs that are not bugs. Deduplication is fingerprint-based, so a false positive stays consistently identifiable rather than reappearing as something new each run.

Playwright errors on install. Almost always a missing system dependency. npx playwright install --with-deps chromium fixes it on Linux and CI images.

Where to go next

• The full open-source overview — what is in the box, and how the CLI compares to the hosted tier.
• What monkey testing actually is — the tiers, and why this technique became viable again.
• Other monkey testing tools — the rest of the field, for web, mobile and code-level fuzzing.
• docs.monkeytest.ai — CLI reference, LLM provider guide, recipes and troubleshooting.
• The repo — issues and pull requests welcome.

If you would rather not install anything, the hosted free tier runs the same engine on our browsers: paste a URL, watch the agent work, get the same report.