FDA commissioner urges health systems to strengthen AI quality oversight
HealthExec December 11, 2024
The critical gap in AI post-market validation
Califf stressed that the biggest concern lies in the post-market phase of AI deployment. Unlike traditional medical devices, AI systems can evolve as they interact with new data, leading to either improved or degraded performance. Without continuous validation, health systems risk relying on algorithms that no longer provide accurate or reliable results.
“It’s clear that particularly with generative AI, you put it in place and then it changes,” he said. “The big deal in AI isn’t the pre-market phase, it’s the post-market phase. You need to continuously validate the model in the environment in which you’re using it. And unfortunately, there’s not a single health system in the U.S. that can do that...