DAComp

Fits teams whose agent spans both pipeline building and analysis, since it keeps the two separate and shows they need different skills. Borrow its split of execution-based grading for engineering tasks and rubric-guided judge grading for analysis tasks.

What it asks the agent to do

• build a multi-stage SQL pipeline from raw sources on an industrial schema
• evolve an existing data system without breaking it
• run an open-ended business analysis and recommend actions
• plan and iterate an exploratory analysis over enterprise tables

What to borrow from it

taxonomygrading design

Caveats

Project page returned 404 and the dataset card points at an anonymous placeholder repo, so only the Hugging Face dataset (MIT) was found; the LLM judge covers the analysis half; engineering success rates are under 20%.

Details

Links

• Paper