GPT-4 with vision has poor accuracy for image-based radiology questions

Medical Xpress September 8, 2024
Elana Gotkine

The large language model GPT-4 with vision (GPT-4V) has high accuracy for text-only radiology questions, but much lower accuracy for image-based questions, according to a study published online Sept. 3 in Radiology.

Nolan Hayden, M.D., from Henry Ford Health in Detroit, and colleagues examined the performance of GPT-4V on radiology in-training examination questions to gauge the model’s baseline knowledge in radiology. The September 2023 release of GPT-4V was assessed using 386 retired questions (189 image-based and 197 text-based) from the American College of Radiology Diagnostic Radiology In-Training Examinations; 377 questions were unique.

The researchers found that GPT-4V answered 65.3 percent of the unique questions correctly, with significantly higher accuracy observed on...