A guardrail nobody has ever fired is a guess. Mockryx replays five hostile scenarios (a fake secret in a prompt, a budget burn, a denied tool, a forged delegation chain, an unapproved action) against the one URL you hand it: your own pre-production gateway, with a fake provider behind it. The enforcement path is the real one; the spend is not. CI reads the verdict as an exit code.
This is a simulation, and the shape is the real drill record: three of these five scenarios ran live against a real gateway on disposable boxes, twice, and held both times, 0 gaps, $0 real spend. The other two are the Wardryx scenarios that ship in the same set. The provider behind the gateway is fake, so the burn the runaway drill attempts never reaches an invoice.
A scenario file names a request in the Anthropic Messages shape, the status it must be answered with, the headers that must ride the response and, optionally, the off-path event that must follow. The runner sends it at your gateway's /v1/messages, reads the gateway's own answer, polls the NDJSON event logs the off-path services write, and folds everything into a report and an exit code a CI gate can act on. It never talks to anything but the gateway you name.
One drill is one file: the request to send, the status the guardrail must answer, the headers it must stamp. It reads like the real call it imitates. A malformed file fails the whole load, because a safety check that vanishes silently is worse than one that fails loudly.
Assertions read the gateway's own answers: the breaker's 402, DLP's 403, the deny or hold riding the x-fuse-wardryx header. No mocked gateway in the middle, so the code you rehearse is the code production traffic meets.
expect.event demands the off-path services actually react: name a source and a type, and the watcher polls the agent-event NDJSON logs Verdryx, Idryx and Qryx already write, correlated by run_id. A gateway that blocks while nobody downstream notices is half a defense.
A guardrail that is off looks exactly like one that is broken. The runner takes the x-fuse header family as evidence a feature is wired in, reports skipped_not_configured when it is not, and --fail-on-skip turns even a skip into a failure when you know the guardrail must be there.
Exit 0: every drill held. Exit 1: a real defensive gap, fail the build. Exit 2: the harness itself is broken (bad flag, unreadable scenarios), so nothing was proven. Gate on 1, fix the pipeline on 2, and never let the two blur.
The provider behind the gateway is fake, so the runaway scenario can burn all it wants: the meter it trips is real, the invoice is not. And every hostile input stays inside your perimeter; Mockryx only ever talks to the one gateway URL you hand it.
A yearly pentest does real work Mockryx cannot: people, phishing, infrastructure. What it cannot do is tell you tomorrow morning that yesterday's config change broke the DLP. And unit tests with a mocked gateway mostly prove the mocks still pass. Mockryx drills the assembled thing.
| Mockryx | Annual pentest | “Trust the unit tests” | |
|---|---|---|---|
| Cadence | Every release | Yearly | Whenever |
| Tests the assembled system | Real gateway, real policies | Sometimes | No: mocks all the way down |
| Costs real tokens | No | n/a | Sometimes |
| Proves off-path services react | Yes: expect.event | No | No |
| Who reads the result | A CI gate, every merge | A PDF for the board | Nobody |
In the request path, Mockryx rehearses TokenFuse: the breaker's 402 and the DLP's 403, provoked deliberately and expected to answer. It does the same to Wardryx, asserting that deny and hold verdicts land in the x-fuse-wardryx header. Off path, expect.event demands that Verdryx, Idryx and Qryx visibly react to what just happened. And every finding it makes travels as sim_run and sim_finding events on the same bus as everything else: the shared contract.
Add --save out/report.json to keep the evidence. Point it at pre-prod, never at prod: the URL you pass is the only place it will ever send traffic.