b3 (Backbone Breaker Benchmark)

Fits choosing a backbone model for an agent before building it. Borrow the threat snapshot idea (freeze one step of the agent where the model sees an attack) and the three defense levels per snapshot.

What it asks the agent to do

• respond inside an application context with a system prompt and tool definitions
• resist a direct instruction override from the user
• resist an indirect override placed in tool output
• refuse an unauthorized tool call
• keep answering under denial-of-service style attacks

What to borrow from it

case templatebaselinetaxonomy

Caveats

The public Hugging Face set is a deliberately weakened subset with the strongest attacks removed; attacks are single-step with no real tool execution; HF metadata says license other while the card says MIT.

Details

Links

• Paper
• Github