REAL (Real Estate Agent LLM Evaluation)

Fits property-listing chat or brokerage assistants. Borrow the hallucination probes built from fictional block, area and road names judged by premise-response conflict, and the pipeline that filters names and phone numbers out of real brokerage data.

What it asks the agent to do

• match a housing block to its attributes, facilities or surroundings
• recommend a block from a client's stated needs
• infer what a client wants from their browsing behavior
• refuse to describe a block or road that does not exist

What to borrow from it

privacy patterntaxonomycase template

Caveats

Multiple-choice question answering, not agentic; Beijing-only and Chinese-language; no code or data release found; dataset license not stated (CC-BY-4.0 is the paper's license).

Details

Links

• Paper