On September 30, 2025, the U.S. Food and Drug Administration (FDA) issued a Request for Public Comment seeking input on “practical approaches to measuring and evaluating the performance of AI-enabled medical devices in the real-world,” including strategies for detecting, assessing, and mitigating performance changes over time (the “Request”). 

The Request acknowledges the opportunities for AI, including generative AI, to improve patient outcomes, advance public health, and accelerate medical innovation.  At the same time, the Request highlights new challenges related to assuring the maintained safety and effectiveness of AI-enabled medical devices across the total product life cycle, and suggests that ongoing, systematic performance monitoring is increasingly relevant for these technologies.

FDA emphasizes that this Request does not signal proposed or final expectations for sponsors of AI-enabled devices, but instead seeks to “advance a broader discussion among the AI healthcare ecosystem on this topic.”  The Agency notes a particular interest in “strategies for identifying and managing performance drift, such as detecting changes in input and output,” but leaves open how FDA plans to incorporate feedback into future regulatory processes and decision-making. 

Key Questions and Themes in FDA’s Request

With a focus on methods that are currently deployed at scale in real-world environments, supported by real-world evidence, and applied in clinical settings, FDA seeks comments on the following topic areas:

1. Performance Metrics and Indicators.  What indicators best measure safety, effectiveness, and reliability?  How should they be defined and weighted?
2. Real-World Evaluation Methods and Infrastructure.  What tools and processes support proactive post-deployment monitoring?  What’s the role of human review vs. automation?
3. Postmarket Data Sources and Quality Management.  Which real-world data sources (e.g., EHRs, device logs, patient-reported outcomes) are most effective?  How do stakeholders address data quality, completeness, and interoperability challenges?  What methods successfully integrate outcomes and feedback into model updates?
4. Monitoring Triggers and Response Protocols.  What triggers deeper evaluation?  How should organizations respond to performance degradation in real-world settings?
5. Human-AI Interaction and User Experience.  How do user behaviors impact performance?  What design, training, or communication strategies help maintain safe use over time?
6. Additional Considerations and Best Practices.  What best practices, barriers to implementation, and incentives have supported these efforts, including to maintain patient privacy and data protections?

How Does the Request Fit Within FDA’s Broader Approach to Real-World Evidence?

Real-world evidence (RWE) is central to the Request.  FDA has invested heavily in RWE policy and science for drugs and devices (indeed, stakeholders are eagerly awaiting the Agency’s finalized device-specific RWE guidance, expected in FY26).  To date, most of FDA’s RWE guidance has focused on one-time studies designed to inform regulatory decisions.  By contrast, the Request emphasizes continuous, real-world performance monitoring, which presents different challenges.  To be clear, although FDA has experience leveraging RWE for postmarket surveillance and pharmacovigilance, FDA’s Request introduces new questions in the RWE space, including whether current evaluation methods are equipped to “predict behavior in dynamic, real-world environments.”

Stakeholders Should Consider Providing Comments to FDA

Comments must be submitted to Docket No. FDA-2025-N-4203 by December 1, 2025.  Covington’s Digital Health team is closely following FDA’s evolving approach to AI oversight.  Please feel free to contact our team for guidance as you evaluate how this Request may impact your organization’s AI monitoring and evaluation strategies.