Mahal Systems TeleQnA
Fits network-operations or telecom customer-support assistants as a knowledge check before agent testing. Borrow its five-category split (lexicon, research overview, research publications, standards overview, standards specifications).
What it asks the agent to do
- answer multiple-choice questions on telecom terms
- answer questions about 3GPP and IEEE standards
- answer questions drawn from telecom research papers
What to borrow from it
baselinetaxonomy
Caveats
Multiple choice only, no tools or actions; questions were machine-generated from documents with human review, so wording can be shallow; standards questions are where models fail; widely reused since 2023 so some leakage into training data is likely.
Details
| Workloads | telecom, knowledge qa |
|---|---|
| Interaction | single turn |
| Environment | static |
| End state checked | No |
| Tools | none |
| Grading | exact match |
| Size | 10,000 |
| Data | mixed |
| Personal data | none |
| License | MIT |
| Harness | own repo |
| Status | active |