Verification - IronBee

When your agent finishes editing code, IronBee runs verification cycles to confirm the change actually works before the task can complete. ironbee install walks you through three setup choices in this order — and each section below shows that install step first, then the command to change it later:

Mode — enforce, assist, or monitoring-only (and, for enforce, how strict to be).
Platforms — which runtime cycles gate a change (browser, node, python, backend, android, terminal).
Project checks — deterministic lint / test / typecheck commands to run first.

The rest of the page covers how the verifier runs, verify-vs-fix, N/A verdicts, and custom scenarios.

Verification modes

IronBee runs in one of three modes, set by two switches: the master toggle (verification enable / disable) and the automatic-enforcement sub-toggle (verification auto enable / disable). The default is assist — the verifier is installed but never blocks completion.

You normally pick the mode while you install — the arrow-key pickers shown in this section are exactly what ironbee install presents (mode, then, for enforce, strictness). Pass --mode enforce|assist|monitor to skip the prompt, or use the verification commands below to switch afterward.

Assist mode (default)

The machinery is installed the /ironbee-verify command, the verifier sub-agent, the devtools MCP server, and permissions so the agent (or you) can verify manually, but nothing is enforced: no verify gate blocks completion, the pre-edit hooks run non-blocking, and the always-on skill/rule are omitted. This is what a fresh install resolves to.

Verification-mode picker with the cursor on assist, described as tools installed but not enforced

Assist keeps verification available but never required — the agent can verify on demand (and every manual cycle is recorded to the Collector) without a gate that blocks task completion. To switch to it later:

ironbee verification auto disable   # assist (default) — manual /ironbee-verify only, nothing enforced

Enforce mode

The agent must pass every active cycle before it can mark a task complete. The blocking verify gate, the always-on skill and rule, the /ironbee-verify command, the verifier sub-agent, and the devtools MCP server are all installed.

Verification-mode picker with the cursor on auto verify (enforce), described as block task completion until changes are verified

Enforce is opt-in — turn on the automatic-enforcement sub-toggle:

ironbee verification auto enable    # enforce — block task completion until every active cycle passes
ironbee verification auto disable   # back to assist (the default)

ironbee verification enable only flips the master toggle; the mode it resolves to still depends on auto (assist unless you’ve also run verification auto enable).

Strict mode

Enforce has one follow-up choice: how strict the gate is about N/A verdicts — the verifier’s way of declaring a change has no runtime surface to exercise (a type-only refactor, a docs tweak). When you pick enforce, install shows a second picker for it right away (assist and monitoring skip it — there’s no gate to enforce). Non-strict (default) lets the agent skip changes with nothing to exercise — refactors, type-only edits, docs:

Picker titled 'How strict should verification be?' with the cursor on non-strict

Strict always makes the agent actually exercise the change — no cycle can be skipped:

Picker titled 'How strict should verification be?' with the cursor on strict

To set it later (rather than at install):

ironbee verification strict enable    # reject all N/A verdicts — every active cycle needs real evidence
ironbee verification strict disable    # back to the default — N/A verdicts accepted (and recorded)

With strict on, a verdict that tries to mark a cycle not-applicable is blocked, and the agent has to produce the cycle’s required tools. Strict only matters in enforce mode (in assist and monitoring-only there’s no gate to enforce it). It’s read live by the gate, so it takes effect on the next session without re-rendering artifacts. Strict maps to the verification.strict config key and accepts the usual -g/--global and --local target flags. For how N/A verdicts work in the first place, see N/A verdicts below.

Monitoring-only mode

The agent is never blocked and none of the verification machinery is installed no enforcement hook, skill, rule, command, verifier sub-agent, or MCP server but session lifecycle, tool calls, and timing still flow to the IronBee Collector. Use this when you want the analytics without changing how the agent works.

Verification-mode picker with the cursor on monitor only, described as no enforcement, only track sessions / activity / tools for analytics

Switch to it (or back) anytime:

ironbee verification disable   # monitoring-only — no machinery, analytics only
ironbee verification enable    # re-enable verification (resolves to assist unless auto is on)

auto only matters while verification is enabled. If verification.enable is false (monitoring-only), the mode is monitoring regardless of auto, and ironbee verification auto … warns that it has no effect until you re-enable verification.

The master and auto toggles accept the same target flags as everything else (-g/--global, --local; default is the project config) and warn when a higher-priority layer is shadowing the layer you wrote to.

Restart your editor or agent session after toggling; the change takes effect on the next session.

Verification platforms

Each platform is a separate cycle with its own tools. IronBee installs all six, but only some are active by default:

Platform	Default	What it verifies	Tool prefix
Browser	On	Frontend changes navigates pages, takes screenshots, checks the console and accessibility	`bdt_*`
Node	Opt-in	Node.js backends attaches the V8 inspector, sets probes, reads runtime snapshots, and captures outbound HTTP	`ndt_*`
Python	Opt-in	Python backends attaches to a running process over debugpy/DAP, sets probes, dumps threads, and captures outbound HTTP	`pdt_*`
Backend	Opt-in	Any backend runtime drives real HTTP, gRPC, GraphQL, or WebSocket calls and checks the responses	`bedt_*`
Android	Opt-in	Android app changes drive an emulator over ADB, launch the app, tap and swipe, take screenshots and UI snapshots, read Logcat, and capture HTTP traffic	`adt_*`
Terminal	Opt-in	CLI / REPL / TUI changes spawns the program in a pseudo-terminal (PTY), sends input, and reads the output the same way `tmux` drives a pane	`tdt_*`

Every enabled cycle’s tools are served by a single ironbee-devtools MCP server (a “compose” server that multiplexes them under their fixed per-platform prefixes). The completion gate still routes each tool call to its cycle by that prefix — so browser-devtools, node-devtools, etc. remain the logical cycle names even though there’s just one physical server. Enabling or disabling a cycle changes which prefixes that one server exposes.

The browser cycle covers most frontend work out of the box. The Node, Python, backend, Android, and terminal cycles stay off until you opt in, so those changes aren’t gated until you ask for it.

Choosing platforms at install

You pick which platforms to verify while you install, from an arrow-key multi-select — space toggles a cycle, a toggles all, Enter confirms. (You can also flip any cycle afterward with the enable / disable commands below.) The picker also offers an s “suggest” key. Press it and IronBee analyzes your project with your client’s model — Claude Code, Codex, and Cursor can all run it (when several are selected it uses the highest-priority one, claude > codex > cursor):

Platform multi-select with the 's suggest (claude)' key highlighted and 'Analysing project with claude…' shown while IronBee inspects the project

…then replaces your selection with the cycles it recommends — still editable before you press Enter:

Platform multi-select after suggestion with browser, node, and backend checked as the recommended set

Suggestion is opt-in — nothing runs until you press s. It runs a one-shot headless prompt through your client (claude -p, codex exec, or cursor-agent -p), so the client must be signed in. If the analysis is cancelled (Esc), times out, or fails, your current selection is kept.

Here’s each platform, and what its row looks like in that picker. Browser (on by default) — a real browser: navigate pages, take screenshots, and check the console and accessibility. It covers most frontend work with no setup, and is the only cycle enabled out of the box.

Platform multi-select with browser checked and the cursor on the browser row

Node — Node.js runtime (tracepoints, logpoints, exceptions, variables, logs) plus outbound HTTP capture (axios, got, node-fetch, fetch/undici, http2, gRPC):

Platform multi-select with node checked and the cursor on the node row

Python — a running Python process over debugpy (the Debug Adapter Protocol): tracepoints, logpoints, exceptions, watches, stdout/stderr logs, an all-thread thread dump (py-spy-style, for deadlock / GIL / hang diagnosis), plus outbound HTTP capture (urllib/http.client, requests, httpx, aiohttp). It attaches to an already-running process without blocking it — an already-listening debugpy at host:port (default 127.0.0.1:5678), a PID (POSIX), a process name, or a docker container — and never modifies the target’s source.

Platform multi-select with python checked and the cursor on the python row

The Python cycle needs debugpy installed in the target’s Python environment (pip install debugpy). Unlike Node’s built-in inspector, debugpy is a third-party package, so IronBee can’t attach without it.

Backend — HTTP, gRPC, GraphQL, WebSocket, DB, and logs:

Platform multi-select with backend checked and the cursor on the backend row

Android — device/emulator drive, taps, swipes, screenshots, UI snapshots, Logcat, HTTP capture:

Platform multi-select with android checked and the cursor on the android row

Terminal — CLI / REPL / TUI driving over a PTY: spawn a program, send keystrokes, capture output and exit codes. There are no screenshots here, so terminal is not a recording cycle.

Platform multi-select with terminal checked and the cursor on the terminal row

Enable or disable a platform

To change platforms after install (rather than re-running it), each has an enable / disable subcommand:

ironbee node enable       # turn on Node.js runtime verification
ironbee python enable     # turn on Python runtime verification (needs debugpy in the target env)
ironbee backend enable    # turn on backend protocol verification
ironbee android enable    # turn on Android device/emulator verification
ironbee terminal enable   # turn on terminal (CLI / REPL / TUI) verification
ironbee browser disable   # turn off the browser cycle (e.g. backend-only projects)

Re-enabling is the inverse: ironbee browser enable, ironbee node disable, and so on. Changes are written to the project config and your client’s hooks/tools are re-rendered automatically. By default these write to the project config (<project>/.ironbee/config.json). To target a different layer:

Flag	Writes to
(none)	Project config committed, shared with your team
`-g`, `--global`	`~/.ironbee/config.json` applies to all your projects
`--local`	`<project>/.ironbee/config.local.json` gitignored, just for you

Enabling a platform applies sensible built-in file patterns automatically; you don’t have to list which files to verify. To fine-tune which files trigger each cycle (custom verify patterns, ignored paths, devtools settings), see Configuration.

Project checks run first (lint, tests, types)

Before any devtools verification runs, IronBee can run your own deterministic project checks — linters, type-checkers, unit tests, a build — as the first step of every verification cycle. In enforce mode, a required check that fails (or never ran) blocks completion, so the agent has to fix it and re-run before it can finish. There are two ways to set them up — let IronBee suggest them (at install, or on demand) or write them by hand. Both land in the same verification.checks config key.

Let IronBee suggest your checks

Beta — the suggested commands are a starting point to review, not a finished config.

IronBee can analyze your project with your AI client and propose concrete checks, then let you approve a subset before it writes them. Like the mode and platform steps, this is offered at install first — the last step of ironbee install, behind a yes/no gate (default No), after the mode and platform pickers. Accept the gate and IronBee analyzes the project with the client’s model (Esc cancels):

Beta yes/no gate accepted, with 'Analysing project with claude… (Esc to cancel)' shown while IronBee inspects the project for check commands

then hands you a per-check approval list, each check tagged required or advisory (from its kind — typecheck and test default to required). Per row, r flips required ⇄ advisory and c flips conclusive (which also forces required):

Per-check approval multi-select showing 'lint — npm run lint · advisory' and 'test — npm test · required' with checkboxes

Optionally, IronBee then validates your picks by running each once (a [y/N] prompt at install, or the --validate flag on the command) — it shows pass/fail and re-presents the list so you can drop any that broke before it writes:

Validate step showing [PASS] lint and [PASS] test, then a confirm-to-save multi-select with each check marked PASS

The model reads your package.json scripts, Makefile, justfile, pyproject.toml, Cargo.toml, go.mod, Gradle, turbo/nx, and the like, and returns the concrete command for each kind — preferring your project’s own existing scripts/targets and never modifying any file. This is “discovery LLM, deterministic runtime” — the model runs only at install (or the command below), never during a verification cycle; the gate always just re-runs the committed commands. The same flow, standalone. Run it anytime afterward — to add checks to a project you already set up, or re-suggest after big changes — with ironbee checks suggest:

ironbee checks suggest                 # analyze the project and propose verification.checks
ironbee checks suggest --validate      # run each proposal once and drop the ones that fail before writing
ironbee checks suggest --yes           # accept the full suggested set (for scripts/CI)

Flag	Description
`--client <name>`	Which client’s model to analyze with. Claude Code, Codex, and Cursor all work; defaults to the highest-priority detected one (`claude > codex > cursor`).
`--validate`	Run each suggested check once and drop the ones that fail before writing.
`-y`, `--yes`	Accept the full suggested set without the approval prompt.
`-g`, `--global` / `--local`	Which config layer to write to (default is the committed project config).

Re-running is additive and non-destructive — a fresh suggestion is unioned with your existing checks (your hand-tuned entries win on a name clash), so you can re-suggest anytime (or re-run install) without losing customizations.

Resolving the suggestion runs a one-shot headless prompt through your client — Claude Code, Codex, or Cursor (cursor-agent) — so the client must be installed and signed in. With more than one available it uses the highest priority (claude > codex > cursor). If none can run headless, write verification.checks by hand instead.

Write the checks by hand

Or set verification.checks directly — a JSON array where each entry is one command:

{
  "verification": {
    "checks": [
      { "name": "lint",      "command": "npm", "args": ["run", "lint"], "kind": "lint" },
      { "name": "typecheck", "command": "npm", "args": ["run", "typecheck"], "kind": "typecheck", "required": true },
      { "name": "test",      "command": "npm", "args": ["test"], "timeoutMs": 120000, "kind": "test", "required": true }
    ]
  }
}

Each entry takes:

Field	Required	Meaning
`name`	yes	Label shown in the verdict and any block message.
`command`	yes	Executable to run.
`args`	no	Argument list.
`env`	no	Extra environment variables.
`cwd`	no	Working directory (defaults to the project dir).
`timeoutMs`	no	Per-check timeout in milliseconds.
`required`	no	Whether a failure blocks completion (see below).
`conclusive`	no	Makes the check the arbiter of the whole cycle — see Conclusive checks. Implies `required`. Hand-set only (never suggested).
`kind`	no	Advisory label — `typecheck` / `lint` / `test` / `build` / `format` / `other`. Cosmetic to the gate; it drives the suggester’s required-by-default choice (`typecheck` and `test` default to required).

How a check gates completion

The verifier runs the checks via ironbee hook run-checks at the start of the cycle, and IronBee trusts the recorded exit codes, not the agent’s word for it. In enforce mode, a required check that exits non-zero or times out blocks completion — the agent has to fix it and re-run — and a required check that never ran blocks too, so the agent can’t quietly skip it. Non-required checks are recorded but don’t gate. A failed check counts toward maxRetries like any other block.

Check results stay local to your machine (recorded under .ironbee/, not sent to the Collector), and verification.checks is read live — editing it takes effect on the next session without re-rendering artifacts. In assist and monitoring-only mode there’s no gate, so the checks run during a manual /ironbee-verify but never block.

Conclusive checks (let a check decide the cycle)

By default a check is a gate — it can block completion, but the agent still has to run the devtools verification (screenshots, probes, HTTP calls) to actually pass. A check marked conclusive: true is stronger: it becomes the arbiter of the cycle, so its exit code alone can conclude verification — no devtools step needed. Use it when a deterministic command (a full end-to-end suite, an integration test) is a trustworthy stand-in for driving the app by hand. conclusive implies required, and there are two outcomes:

All conclusive checks pass → ironbee hook run-checks auto-submits the pass verdict and closes the cycle. The devtools verification is skipped entirely — the recorded exit codes are the evidence, so this is strict-compatible (a machine result, not an agent claim). The agent just reports and stops.
A conclusive check fails → a deferred fail: instead of stopping at the red command, the verifier turns to the devtools tools to diagnose the named failure, then submits a findings-rich fail verdict. IronBee auto-prepends a conclusive check "<name>" failed (exit N) issue and refuses a pass or N/A for that cycle until a fresh cycle re-runs the checks green.

Because it can replace the devtools cycle, conclusive is hand-set only — the suggester never marks a check conclusive. Set it in the config by hand, or flip it in the approval pickers (at install, ironbee checks suggest, or the TUI) with the per-row c key (c toggles conclusive — and forces required; r toggles required/advisory). Give a long conclusive suite an explicit timeoutMs (checks default to 120s).

{
  "verification": {
    "checks": [
      { "name": "e2e", "command": "npm", "args": ["run", "test:e2e"],
        "timeoutMs": 600000, "kind": "test", "conclusive": true }
    ]
  }
}

Conclusive checks only conclude the cycle in enforce mode (assist and monitoring-only never gate). A deferred fail counts toward maxRetries and releases at the cap with the failure reported, like any other block.

How verification runs (the verifier sub-agent)

On Claude Code and Codex, IronBee doesn’t let the main agent drive the devtools tools directly. Instead it installs a dedicated ironbee-verifier sub-agent that owns those tools (plus read-only Read / Grep / Glob so it can understand the change it’s verifying — but no edit tools), and the main agent delegates the verification cycle to it — automatically at the completion gate, or manually via /ironbee-verify. The verifier runs every active cycle, submits the verdict, and hands back a short summary. The point is to keep the heavy devtools output (DOM snapshots, console logs, screenshots) inside the sub-agent’s context instead of the main conversation. The sub-agent shares the session, so its events still flow to the Collector and its token/tool usage is folded into the session analytics.

Cursor has no verifier sub-agent — its main agent runs the verification itself, because Cursor’s sub-agents can’t share the verification session.

Picking the verifier model

By default the verifier runs on the same model as the main conversation. Pin it to a specific model — for example to verify on something cheaper or faster than your main coding model — with ironbee verification model:

ironbee verification model sonnet --client claude   # pin Claude's verifier
ironbee verification model gpt-5.5 --client codex    # pin Codex's verifier
ironbee verification model sonnet                    # apply to every client (bare string)

--client is optional when exactly one verifier-capable client is installed and required when there are two (Claude + Codex), since one model name rarely works for both. To go back to inheriting the session model:

ironbee config unset verification.model

The setting accepts the usual -g/--global and --local target flags and re-renders the verifier artifact when it changes. It maps to the verification.model config key. Cursor is a no-op.

On Codex the verifier needs some resolvable model. It inherits your ~/.codex/config.toml model; if you haven’t set one there, pin verification.model so the sub-agent can spawn.

Verify-only vs. fix mode

When you trigger /ironbee-verify by hand (or $ironbee-verify on Codex), an optional leading word picks what happens on a fail verdict:

/ironbee-verify              # verify-only (default) — report the verdict and stop
/ironbee-verify fix          # fix-and-re-verify — on a fail, fix the issues and loop until it passes
/ironbee-verify fix log in as admin, open /billing   # fix mode + a custom scenario

Mode	Leading word	On a fail verdict
Verify-only (default)	(none) or `report`	Runs the cycle, submits the verdict, reports it, and stops. It never edits your code, even when the verdict is a fail — use it to check the current state.
Fix	`fix`	On a fail, the agent fixes the reported issues and re-verifies, looping until it passes (or hits `maxRetries`).

In enforce mode the choice is enforced mechanically, not just suggested: a fix run won’t let the agent stop on a fail without addressing it, while a verify-only run is allowed to report a fail and end the turn. A custom scenario can follow the mode word.

This only changes the manual command’s behavior. The automatic completion gate is unaffected — when the agent edits code and tries to finish, it must still pass every active cycle. In assist mode nothing blocks the agent, so fix there is advisory; in monitoring-only mode the command isn’t installed.

When a change has nothing to verify (N/A verdicts)

Some edits match a cycle’s file patterns but have no runtime surface to exercise — a type-only change, a behavior-neutral refactor, a config or docs tweak. Rather than fake screenshots or HTTP calls to satisfy the gate, the verifier can submit a not-applicable (N/A) verdict, declaring there’s nothing to verify and explaining why:

Global N/A — no active cycle applies to the change. The verifier reports status: not_applicable with a short reason, and the gate lets the task complete without any tool evidence.
Per-platform N/A — some cycles apply and some don’t. The verifier verifies the ones that do (a normal pass/fail) and exempts the rest by name, with a reason for each exemption.

N/A verdicts are recorded and observable — they show up in the session’s verdict like any pass or fail, with the exempted cycles and reasons attached, so you can see why a change wasn’t exercised. To guard against abuse, the gate refuses an N/A after a previous fail on the same change (an agent can’t downgrade a real failure to “not applicable” to escape the gate). To refuse N/A entirely — demanding real tool evidence for every active cycle — turn on strict mode (the enforce follow-up choice covered under Verification modes). It’s picked at install right after enforce, or toggled later with ironbee verification strict enable.

N/A is off-limits after a fail and under strict mode — it’s an escape hatch for changes that genuinely have no runtime behavior, not a way around a real verification.

Custom verification scenarios

By default the agent verifies the areas affected by what changed. To tell it exactly what to exercise, pass a custom scenario to /ironbee-verify (or $ironbee-verify on Codex) — either inline or as a path to a file:

/ironbee-verify log in as admin, open /billing, confirm the invoice total updates
/ironbee-verify ./scenarios/checkout.md

When a scenario is supplied it’s authoritative: it replaces the default “exercise the changed pages” guidance, and the verifier drives precisely the flows, states, and endpoints it names. A scenario file is read at run time (any location, any format), so you can keep reusable test scripts in your repo and point at them. The completion gate is unchanged — every active cycle’s required tools still have to run for a pass. To capture a flow once and replay it by name instead of re-typing it every time, save it as a scenario and pass /ironbee-verify scenario:<name>.

What’s next?

Saved scenarios

Capture reusable verification flows and replay them by name.

Verification context

Give the agent area-specific instructions with .ironbee/VERIFICATION.md files.

Configuration

Fine-tune which files trigger each cycle, retry limits, and more.

Inspecting sessions

Check a session’s verdict and debug why it passes or fails.

​Verification modes

​Assist mode (default)

​Enforce mode

​Strict mode

​Monitoring-only mode

​Verification platforms

​Choosing platforms at install

​Enable or disable a platform

​Project checks run first (lint, tests, types)

​Let IronBee suggest your checks

​Write the checks by hand

​How a check gates completion

​Conclusive checks (let a check decide the cycle)

​How verification runs (the verifier sub-agent)

​Picking the verifier model

​Verify-only vs. fix mode

​When a change has nothing to verify (N/A verdicts)

​Custom verification scenarios

​What’s next?

Saved scenarios

Verification context

Configuration

Inspecting sessions

Verification modes

Assist mode (default)

Enforce mode

Strict mode

Monitoring-only mode

Verification platforms

Choosing platforms at install

Enable or disable a platform

Project checks run first (lint, tests, types)

Let IronBee suggest your checks

Write the checks by hand

How a check gates completion

Conclusive checks (let a check decide the cycle)

How verification runs (the verifier sub-agent)

Picking the verifier model

Verify-only vs. fix mode

When a change has nothing to verify (N/A verdicts)

Custom verification scenarios

What’s next?