Mahal Systems IntellAgent
Fits any policy-bound chat agent whose team wants to generate their own test cases from a prompt or policy document. Borrow the policy graph (policies as nodes, co-occurrence and difficulty as edges) used to sample multi-policy scenarios, and the critique step that maps failures back to specific policies.
What it asks the agent to do
- handle a generated edge-case customer scenario that combines several policies
- act on a database through tools while staying within the stated rules
- complete the conversation without violating any of the policies the scenario tests
What to borrow from it
user simulatorcase templatetaxonomy
Caveats
A generator and framework, not a fixed dataset, so there is no stable number to compare against; grading is an LLM critic rather than a state check; optimization features are paid, and recent maintenance is unclear.
Details
| Workloads | customer support, travel, education, tool calling |
|---|---|
| Interaction | multi turn simulated user |
| Environment | api sandbox |
| End state checked | No |
| Tools | the agent under test's own tools; example airline environment with a database |
| Grading | llm judge |
| Size | Not published |
| Data | synthetic |
| Personal data | none |
| License | Apache-2.0 |
| Harness | own repo |
| Status | active |