GAIA

Fits research-style assistants that must return one checkable answer. Borrow the short-answer exact-match format and the three difficulty levels.

What it asks the agent to do

• answer a question that needs web browsing and several reasoning steps
• read an attached file (spreadsheet, PDF, image, audio) to find the answer
• combine facts from several sources into one short answer

What to borrow from it

grading designbaselinecase template

Caveats

Read-only question answering, no state changes; top agents now score high; answers to 300 test questions are private and the dataset is gated.

Details

Links

• Paper
• Github
• Leaderboard