MultiHop-RAG

Fits retrieval-augmented agents over a corpus where one question spans several documents. Borrow its split of retrieval scoring (did the right evidence come back) from answer scoring, and its four query types including null queries with no answer in the corpus.

What it asks the agent to do

• answer a question that needs evidence from 2 to 4 news articles
• compare or order facts reported by different sources
• answer a date-ordering question across articles
• return null when the corpus has no answer

What to borrow from it

grading designcase template

Caveats

Corpus is 609 English news articles, so it is small and single-domain; answers are kept short (yes/no, entity, before/after) to allow simple accuracy, which limits what it says about long-form answers; GPT-4 was used in dataset validation, so some label bias toward it is possible.

Details

Links

• Paper
• Github