CMU Researchers Help Close a Critical Security Gap Across AI Platforms

An AI flaw that can be found today in one model could be quietly replicated across dozens of products and services built on the same underlying technology. Until now, the AI community has lacked a formal pathway to report those flaws, alert affected vendors and coordinate a response. Researchers at Carnegie Mellon University’s Software Engineering Institute(opens in new window) (SEI), alongside collaborators from academia and industry, helped build one. Flaw Reporting for AI(opens in new window) (FLARE-AI) is a new open-source platform that lets anyone report an AI vulnerability and route it to developers, vendors and government agencies equipped to act. 

Closing the AI security gap

Without a formal reporting structure, it is likely that many AI flaws and vulnerabilities have gone unreported, according to Lauren McIlvenny(opens in new window), technical director of threat analysis at the SEI and an adviser to the FLARE-AI project. When flaws are reported, they are often sent to a single vendor.

“A reporter might spot a problem in a particular model or system, but they’re not looking across all the vendors and third-party integrators to see if they share the same structural weakness,” McIlvenny said. 

Using the FLARE-AI website, anyone can complete a form that generates a standardized, machine-readable report about an AI flaw, vulnerability or incident. The user can then tell the system to submit the report to independent third parties such as the SEI, government agencies, incident databases, AI model-hosting platforms or AI model developers. From there, the receiving organization can address the flaw directly or coordinate disclosure and remediation among affected vendors.

Bringing proven cybersecurity practices to AI

The system mirrors longstanding mechanisms for reporting software vulnerabilities, such as the Vulnerability Information and Coordination Environment (VINCE)(opens in new window) platform run by the SEI’s CERT Coordination Center (CERT/CC)(opens in new window).

VINCE is one of the reporting pathways that FLARE-AI connects to, thanks to the back-end work of Greg Strom, an SEI software engineer who was also an adviser to the project. Once VINCE receives a flaw report from FLARE-AI, experts from the CERT/CC and the SEI’s AI Security Incident Response Team (AISIRT)(opens in new window) will review the report and, if warranted, arrange disclosure with affected developers, vendors and integrators.

“By integrating FLARE-AI into VINCE and our coordinated vulnerability disclosure process, we’ll be able to provide cross-platform examination,” said McIlvenny. “We can issue CVE (Common Vulnerabilities and Exposures) IDs and vulnerability notes to make sure all affected tool vendors, third-party integrators and users know about the flaw. That’s what traditional cybersecurity processes are going to bring to the table.”

Recent federal initiatives(opens in new window) have called for stronger coordination around AI vulnerability discovery, validation and remediation, including the creation of an AI cybersecurity clearinghouse. FLARE-AI and the organizations connected to it, including CERT/CC, are positioned to be links in that reporting chain. 

Community approach, collective security

FLARE-AI is the culmination of work by a group of AI and security researchers from academia, industry and non-profits. The seeds for the system were planted at a 2024 workshop on the future of third-party AI evaluation(opens in new window), where participants called for standardized AI flaw reports, AI disclosure programs and improved infrastructure for sharing flaw information. 

McIlvenny, who spoke at the workshop and coauthored the resulting paper(opens in new window), advised the FLARE-AI team on adapting cybersecurity and reporting practices to the emerging AI vulnerabilities she sees through her work leading AISIRT. 

The collective approach to AI security is critical for the field, said McIlvenny. 

“The whole community recognizes the importance of AI. Now AI researchers need to come into the security field, learn what they can and change it where it needs to be changed,” she said.

As a trusted partner for government, industry and academia, the SEI has long worked to bring together(opens in new window) the often disconnected AI and cybersecurity communities. Merging these disciplines represents an important step in the maturity of AI, one experienced by previous technology innovations such as industrial control systems and cloud computing, said McIlvenny. 

“The AI community is starting to learn the cybersecurity processes that they could adopt or adapt. FLARE-AI is a place for them to find out how AI flaws fit into the world of coordinated vulnerability disclosure.”