A study recently published in JAMA, conducted by computer scientists and healthcare professionals from the University of Michigan, investigated the role of artificial intelligence in diagnosing patients in hospitals. The focus of the study was to understand the impact of clinicians being aware of the workings and potential biases of AI models on diagnostic precision.
The study explored whether visual explanations of AI models could assist healthcare providers in identifying algorithms with potential systematic biases, which might lead to inaccuracies. However, the findings indicated that these explanatory tools were ineffective in aiding clinicians to spot biased AI models.
To evaluate the influence of systematic bias in AI on diagnostic accuracy and to see if visual explanations of AI models could correct errors, the researchers organized a randomized clinical vignette survey. This survey involved hospitalist physicians, nurse practitioners, and physician assistants from 13 states in the U.S.
Participants were presented with nine clinical vignettes of patients admitted with acute respiratory failure. Each vignette included symptoms, physical examination findings, lab results, and chest x-rays. The clinicians’ task was to assess the probability of pneumonia, heart failure, or chronic obstructive pulmonary disease being the primary causes of each patient’s acute respiratory failure.
Initially, clinicians analyzed two vignettes without AI assistance. Subsequently, they were divided into groups to review six vignettes with AI input, with some receiving additional AI model explanations. Half of these vignettes were based on standard-model predictions, while the others were based on systematically biased predictions.
The study revealed that diagnostic accuracy improved by 4.4% when clinicians used standard AI model predictions along with explanations, compared to the baseline. However, when faced with systematically biased AI predictions, accuracy dropped by over 11%, and the presence of model explanations did not counteract the negative impact of these inaccurate predictions.
The study concluded that while standard AI models can enhance diagnostic accuracy, systematic biases can diminish it, and the typical visual explanations provided by AI models are ineffective in preventing this detrimental effect.
The most significant role of Artificial Intelligence in healthcare is its ability to process vast amounts of data and identify patterns that might otherwise go unnoticed by human clinicians.