> This page is for Agent Handler.

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.merge.dev/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.merge.dev/_mcp/server.

# Rule Tester

> How to use the Agent Handler Rule Tester to validate custom regex rules, Standard Entity Rules, and AI Guardrails before going live.

The tester lets you paste a sample input and see exactly which rules match, which entities are detected, what each match's confidence score is, and what each [AI Guardrail](/merge-agent-handler/secure/ai-guardrails) decided. Use it before saving any new rule and any time you change a threshold or context keyword on an existing one. Most rule mistakes look correct in the editor and only show up once real traffic hits.

Open [Security → Guardrail tester](https://ah.merge.dev/security/guardrail-tester).

## How to use it

1. Paste a sample of the data you want to test against. A real tool argument or tool result, copied from the [Tool Call Logs](/merge-agent-handler/observe/tool-call-logs), is the best source.
2. Pick which rules to test against, every active rule by default, or a subset you're tuning.
3. Click **Run**.

The tester streams the result as the pipeline runs and returns:

* A list of every match, with the entity type, the matched substring, the confidence score, and the rule that produced it.
* Whether each match would clear the threshold (and therefore fire its action) or be suppressed.
* For matches near context keywords, which keywords contributed to the score.
* Each guardrail's verdict, pre-run and post-run, and whether the call would have been blocked.

A library of sample inputs covering payment, health, personal, secret, financial, and legal content is built in, along with a benign control for confirming a new rule doesn't fire on ordinary traffic.

## What to test

Run every new rule through three cases.

### Positive cases

The data the rule is supposed to catch. Confirm the score lands above your threshold.

```
Sample:  Customer CUST-00547 is requesting a refund.
Expect:  CUSTOMER_ID matched on "CUST-00547", confidence 0.85, fires.
```

### Negative cases

Data that looks similar but shouldn't match. Confirm the score lands below your threshold (or doesn't match at all).

```
Sample:  The CUSTODIAL bank holds the assets.
Expect:  No match (CUST- pattern requires dash and digits).
```

### Edge cases

The fuzzy boundaries. Words near context keywords. Different casings. Whitespace variants.

```
Sample:  The customer ID is 12345.
Expect:  Depends on rule - if you intended this to match, you need a more flexible pattern.
```

## Tuning thresholds with the tester

The score the tester returns is the score the rule will see in production. To dial in the threshold:

1. Run a representative sample of legitimate (false-positive) matches. Note the highest score they produce.
2. Run a representative sample of real (true-positive) matches. Note the lowest score they produce.
3. Set the threshold above the false-positive max and below the true-positive min.

If those two numbers cross - false positives score higher than true positives - your rule needs better disambiguation: more specific pattern, more context keywords, or a different approach entirely.

## Guardrail verdicts in the tester

The tester runs one sample through the whole Gateway, so it covers [AI Guardrails](/merge-agent-handler/secure/ai-guardrails) as well as the pattern rules. Results stream back in the order the Gateway produces them:

1. Entity and custom-rule detections on the sample
2. One verdict per pre-run guardrail, as each evaluation lands
3. The outcome, meaning whether the call would have been blocked
4. One verdict per post-run guardrail

A guardrail row shows whether it matched and the content the evaluator pointed at, and a row marked as an evaluation failure means the evaluator returned nothing usable for that guardrail rather than clearing it. Rerun the sample before reading anything into a failed row.

## Testing context keywords

The tester shows which context keywords fired on each match. Useful when you can't tell why a rule is or isn't matching:

* Score lower than expected? Your context keywords aren't appearing in the test sample.
* Score higher than expected? Common words near matches are inflating scores. Tighten the keyword list.

## After deploying

The tester is for pre-flight validation. The [Alerts dashboard](/merge-agent-handler/secure/violations-and-alerts) is for post-flight observation. After you push a rule live, watch the breakdown for a few hours - if the volume looks wrong, come back to the tester with real samples from the violation log.

## Next

Watch what your rules catch in production via [Violations and alerts](/merge-agent-handler/secure/violations-and-alerts).