Enhancing Clinical Multiple-Choice Questions Benchmarks with Knowledge Graph Guided Distractor Generation

Authors: Running Yang (Equal contribution), Wenlong Deng (Equal contribution), Minghui Chen, Yuyin Zhou, Xiaoxiao Li

Published in: ICML 2025 Workshop on Reliable and Responsible Foundation Models

Overview

Knowledge Graph Guided Distractor Generation (KGGDG) follows medical knowledge-graph paths to construct plausible but incorrect answers to clinical multiple-choice questions. Evaluations on six medical QA benchmarks show that these answer choices expose weaknesses that easier distractors can conceal, supporting more demanding assessments of medical LLMs.