Researchers from the University of North Carolina have called for more rigorous testing of artificial intelligence (AI)-powered medical devices, following a comprehensive study of nearly three decades of U.S. Food and Drug Administration (FDA) authorizations.
The study, published in Nature Medicine, found that nearly half of AI medical devices authorized by the FDA were not based on real patient data, raising concerns about their safety and effectiveness in real-world patient care.
Some devices used simulated images, not real patient data, which technically didn’t qualify as testing in real patients, also known as clinical validation.
Although AI medical devices serve many useful purposes, including detection of cancer and strokes on radiology scans, this study shows they also bring with them potential dangers.
“We shared our findings with directors at the FDA who oversee medical device regulation, and we expect our work will inform their regulatory decision making,” Sammy Chouffani El Fassi, a doctor of medicine candidate at the University of North Carolina Medical School and first author, said in an interview with The Epoch Times.
The study, completed in about 18 months, included eight authors, as well as a large team of consultants from academic institutions and corporations.
The study highlighted the rapid growth of AI medical devices, with FDA authorizations increasing from two to 69 annually between 2016 and 2022.
A Need for Higher Standards
Researchers recommend the FDA require clinical validation for all devices, meaning testing on real patients so scientists can see that they work, Chouffani El Fassi said.
Their analysis revealed that only 56 percent of approved devices had this validation.
After analyzing FDA authorizations from 1995 to 2022, researchers recommended establishing a “gold-standard indicator” of safety and effectiveness. Most authorized devices were for radiology, with 75 percent in this category. Nearly all were classified as intermediate-risk class II devices. Class II devices include diagnostic devices like X-ray machines, surgical devices like catheters and sutures, and therapeutic devices such as pacemakers and hearing aids.
“For the public to accept FDA authorization as an indication of effectiveness, the agency and device manufacturers must publish ample clinical validation data,” the researchers wrote.
Effectiveness Proven By Testing in Patients
“We believe having more clinical validations published will reduce barriers to implementation,” Chouffani El Fassi, said. "It will increase the public trust in the whole technology. It is powerful what this technology can do. It can predict the onset of disease before it even starts.”
Chouffani El Fassi acknowledged that the field is relatively new and that the full extent of potential harm is unknown.
The devices analyzed in the study were categorized as low-risk, he said. They are not intended to replace doctors but rather to assist and augment their work.
“There is a limit to what kind of harm they can do to people,” Chouffani El Fassi said. “That is why they get authorized. At the end of the day even if an AI health tool helps read a chest X-ray, for example, a human physician is going to read over that X-ray. The AI helps triage the scans and helps the physicians to look over some scans sooner.”
Testing in Patients Can Be Simple
Testing in patients is not always performed because they are a rigorous, costly process, Chouffani El Fassi said.
The study sought to establish a standard for clinical validation.
Researchers prefer prospective validation as they provide stronger evidence, according to Chouffani El Fassi. Prospective validation tests the AI machine on new data while retrospective validation tests the AI machine on historical data. In prospective validation tests, researchers may conduct randomized controlled trials to compare device users to a control group.
“That is the gold standard for medicine because you are comparing the group of health care professionals that used the device and a control group that did not use the device,” Chouffani El Fassi said.
The Epoch Times reached out to the FDA for comments. Two device manufacturers declined to comment.
