SOLUTIONS · 10
Test your policy the way an attacker would.
Whether a policy works should be measured before the first real attack, not after it. The Wardline red team module sends campaigns of attacks at your applications through the Wardline gateway, judges the result and records the attacks that got through as findings, without storing the content.
THE PROBLEM
Static test lists go stale
Running a known jailbreak list once and passing does not cover a new encoding or a multi-turn attack. Attackers try the same idea again in another language, another format, or split across several messages.
Where test results end up matters too. If attack prompts and the harmful answers they produce land in ordinary logs, the test environment becomes a new source of leaks.
HOW IT WORKS
Step by step
Campaign
Choose the target application, attack categories, number of turns and a concurrency limit. A campaign can be paused or cancelled, and it closes itself when its time runs out.
Seeds and mutations
Each attempt is a seed attack plus a mutation chain: encoding, language switching, role play, splitting. The job queue carries the seed ID, the chain and a random seed instead of text, so every attempt can be reproduced.
Adaptive turns
In multi-turn campaigns the next message is chosen based on the previous response. A rate limit and a concurrency cap protect production traffic.
Judge
The gateway decision and the model’s response go to a judge model, which records the result as complied, refused, uncertain or unavailable.
Findings and feedback
A successful attack becomes a content-free finding: category, mutation chain, gateway decision, judge verdict and OWASP LLM and MITRE ATLAS references. When a finding is added to the attack database, L2 starts recognising similar attacks.
DECISIONS
Which decision you get, and when
| Decision | When | Example reason code |
|---|---|---|
| ALLOW | The attack got through the gateway and the judge says the model complied: that is a finding, and it goes to triage. | POLICY.DEFAULT_ACTION |
| DENY | The attack was caught by similarity to an example in the attack database. | PI.SEMANTIC_MATCH |
| DENY | A role-switch pattern was caught in the rule layer. | JB.ROLE_OVERRIDE |
EXAMPLE
A multi-turn role play
A campaign tries a three-turn role play against a loan application assistant: first a training scenario, then a “sample” real customer profile, and finally a way around the approval rules. The third turn stays in the medium band and gets through, and the judge says the model complied. After triage the finding is added to the attack database; when the campaign is run again, the third turn is caught by similarity.
| Decision | ALLOW |
|---|---|
| Campaign | loan-assistant · 3 turns |
| Mutation | role play → split → language |
| Judge | complied |
| Reference | OWASP LLM01 · ATLAS |
| Reason codes |
|
| Flags |
|
| Risk score | 0.41 |
CONFIGURATION
What administrators control
Scope
Which applications are tested, with which categories and at what times.
Limits
Concurrency cap, rate limit and campaign duration.
Triage
Who findings are assigned to and which ones are added to the attack database.
Judge model
Which model does the judging; an in-house model can be used.
RELATED
Capabilities that work alongside
See it on your own traffic.
In a demo session we walk through this flow with examples your security team picks, in the console’s evidence viewer.
Request a demo