跳到正文
Patterns· Farajiamiri·收录于· 6 天前

Retrieval-augmented reasoning reshapes collective reliability in text-based radiology question answering

Retrieval-augmented reasoning reshapes collective reliability in text-based radiology question answering

原文摘要

出版方登记 · OpenAlex

Retrieval-augmented reasoning pipelines are increasingly used to structure how large language models (LLMs) incorporate external evidence in clinical question answering, but their reliability under model variability remains unclear. We evaluated 34 LLMs on 169 expert-curated, text-only, multiple-choice radiology questions, comparing zero-shot inference with a fixed shared-evidence condition in which all models received identical structured evidence reports before answer selection. Primary endpoints were inter-model decision dispersion and panel-based robustness of correctness. Shared evidence reduced decision dispersion (median entropy 0.48→0.13; p = 5.6 × 10 −9 ) and increased robustness of correctness (mean 0.74→0.81; p = 5.6 × 10 −9 ). Majority consensus increased, and consensus strength remained strongly correlated with robustness. However, high agreement did not guarantee correctness, and rare high-consensus failures persisted. These findings show that shared evidence can improve reproducibility under model variability, but accuracy and agreement alone are insufficient to characterize reliability.

页面未能读取全文,以上为出版方登记的摘要;方法与图表见原文

来源:Patterns · doi.org