Chatbots Fail Standard Cognitive Test
MedPage Today December 18, 2024
— Large language models show susceptibility to cognitive impairment
President-elect Donald Trump may have once scored a perfect 30/30 on the Montreal Cognitive Assessment (MoCA), but artificial intelligence (AI) chatbots didn’t perform nearly as well.
On the well-known cognitive screen, most chatbots — also known as large language models (LLMs) — showed signs of mild cognitive impairment.
ChatGPT 4 and Claude 3.5 each scored 25 points, while Gemini 1.0 scored 16 points, reported Roy Dayan, MD, of Hadassah Hebrew University Medical Center in Jerusalem, and co-authors.
Only ChatGPT 4o achieved a score indicating normal cognition (26 points), the researchers said in The BMJ Christmas issue, an annual collection of light-hearted feature articles and original, peer-reviewed research.
“Colossal advancements in the...