FDA and AI-enabled medical devices: a few statistics
On August 7, 2024, the U.S. Food and Drug Administration updated its comprehensive list of AI-enabled medical devices [1]. It reveals that a total of 950 such devices have now been authorized for use within the United States. This represents a substantial increase in the number of AI-powered medical technologies available to healthcare providers and patients compared to 2022 agency’s update, in which they listed “only” 502 positions.
Image 1. The increasing number of FDA-approved AI-enabled medical devices 2022-2024
The landscape beyond the numbers
However, important questions remain. How does this increase translate into therapeutic fields? How about the impact of the use of AI in medicine? Is it increasing as well? Are AI-enabled medical devices truly improving patient outcomes and reducing costs? How many manufacturers report AI-related adverse events?
By exploring these questions, we can gain a deeper understanding of the implications of this constant increase in AI-enabled medical devices. Additionally, it is also an opportunity to get a clearer picture of the AI landscape hidden behind the statistics.
AI in medicine. Is radiology still dominant?
AI-powered medical image analysis has generated a massive body of literature, with tens of thousands of papers published annually. Radiology has been a pioneer in leveraging technology for image analysis. Its image-centric nature makes it a prime candidate for AI applications. Moreover, it generates a vast amount of data daily, providing rich training material for AI algorithms. Finally, the development of medical imaging software heavily relies on pattern recognition, a topic we covered in a previous post.
Given the recent update to the list, has any other medical discipline taken over radiology’s leading position?
Image 2. AI-enabled medical devices approved by FDA in 2024 grouped by lead panel
Evidently, radiology still holds the top spot. However, apart from thousands of papers dedicated to AI in the medical imaging area there is also a different side of the same coin. A growing body of research suggests that many AI products may not be safe or effective for clinical use.
FDA-approved AI-enabled medical devices vs real patient data
The FDA’s list of approved AI-powered medical devices is to provide transparency. Consequently, the approval process ensures these devices are safe and effective. Moreover, this includes an assessment of their performance across diverse populations.
Recently, a multi-institutional team of researchers has evaluated how exactly AI and algorithmic technologies are being approved for use in patient care. You may read about their findings in Nature Medicine [2]. What’s important here is the fact that despite FDA authorization, AI device manufacturers may overstate the clinical efficacy of their products. This is possible because clearance doesn’t always involve rigorous testing with real patient data. Researchers investigated 521 AI-enabled medical devices authorizations. 144 devices were retrospectively validated. 148 prospectively and only 22 underwent randomized controlled trials. Over 40% of them had no published clinical validation data.
AI-enabled medical device manufacturers experience
Additionally, the researchers of aforementioned paper highlighted that the FDA’s guidance falls short in providing clear distinctions between different types of clinical validation studies for AI medical device manufacturers. This ambiguity can lead to inconsistencies in the level of evidence required for regulatory approval, potentially compromising the safety and efficacy of these devices. And somehow it is confirmed also by the statistics.
The FDA annually receives over two million medical device reports of adverse events [3]. However, the reporting system remains suboptimal. There are factors like lack of recognition, knowledge, and simply: a culture of nonreporting. Moreover, AI-enabled medical devices introduce new risks, necessitating a robust safety monitoring environment to detect and report AEs effectively. To ensure patients’ safety, we need a robust safety monitoring environment, detecting and reporting adverse events specific to these devices.
Safety in the service of the proliferation of medical AI
Recently we familiarised ourselves with the project of the first systematic review comparing adverse events reporting systems for AI as medical device [4]. This research will hopefully shed light on the current state of medical AI safety monitoring, informing regulators, policymakers, and industry stakeholders. The results are expected to be published in Q4 of 2024. We’ll be sure to provide more details as they become available.
References:
[2] Chouffani El Fassi, S., Abdullah, A., Fang, Y. et al. Not all AI health tools with regulatory authorization are clinically validated. Nat Med (2024). https://doi.org/10.1038/s41591-024-03203-3
[4] Kale A.U., Dattani R., Tabansi A., Hogg H.D.J., Pearson R., Glocker B., Golder S., Waring J., Liu .X, Moore D.J., Denniston A.K. AI as a Medical Device Adverse Event Reporting in Regulatory Databases: Protocol for a Systematic Review. JMIR Res Protoc 2024;13:e48156 doi: 10.2196/48156