AI-Powered Vision System Helps Drones Navigate Safely
Media Inquiries
The skies are getting crowded. Every day, the U.S. Federal Aviation Administration manages tens of thousands of flights — from drones and helicopters to commercial jets. As air traffic density increases, so does the need for smarter systems to keep these vehicles from colliding.
Researchers from Carnegie Mellon University’s Robotics Institute(opens in new window) (RI) have developed an AI-powered vision system that helps drones safely detect and avoid other aircraft in crowded skies, without relying on bulky, expensive sensors. ViSafe(opens in new window) makes it possible for lightweight, uncrewed aircraft to fly safely alongside planes and helicopters, paving the way for future delivery drones, emergency responders and other autonomous vehicles to share the airspace.
Sebastian Scherer(opens in new window), associate research professor at RI, said the system could eventually help unlock routine, trusted autonomy in shared skies.
“With airspace growing more crowded, ViSafe shows how AI-driven safety augmentation can enable scalable, certifiable autonomy in the national airspace, and marks an early step toward deploying AI in safety-critical domains,” he said.
Existing technologies to help prevent midair collisions, like automated detection systems and traffic management tools, work reasonably well for large aircraft. But they often rely on bulky, expensive sensors, making them a poor fit for the smaller, lightweight drones that are becoming more common.
Recently graduated RI Ph.D. student Jay Patrikar explained that most of the burden for detecting and avoiding collisions currently falls on pilots.
“This task is already difficult, and becomes even more challenging for small uncrewed aerial systems, which lack onboard pilots and often operate without the heavy sensors used on manned aircraft,” he said.
Enter ViSafe, a vision-only airborne collision avoidance system to equip UAS with “see-and-avoid” capabilities. Patrikar and a team from RI’s AirLab(opens in new window) created the system to be passive, infrastructure-free and generalizable to a wide range of aircraft. It builds on their previous work, AirTrack(opens in new window), which uses high-resolution detection and tracking networks to detect aerial objects. ViSafe expands the initial AirTrack technology to multi-camera settings and uses control barrier functions to nudge pilots before they veer too close to danger.
“Control barrier functions keep systems safe in real time by gently adjusting actions only when needed, ensuring they stay within safe limits without being overly conservative,” said Parv Kapoor, Ph.D. student in the Software and Societal Systems Department(opens in new window) at CMU. “The fast, minimal adjustments help systems stay close to the original control.”
To bridge the gap between theoretical formulations and real-world behavior, the researchers prepared extensive testing for ViSafe, which included developing a “digital twin” of the ViSafe system using the NVIDIA Isaac Sim(opens in new window). “The simulation environment allowed comprehensive testing of ViSafe’s perception abilities across diverse scenarios,” said Ian Higgins, a RI master’s student.
Following successful simulation testing, the researchers moved ViSafe to the field to see if the system held up under real-world conditions. Over the course of approximately 80 hours, the team used the simulation testing results to create representative real-world configurations to analyse the collision avoidance capabilities.
“It’s one thing to show results in a clean, controlled lab setting — but demonstrating consistent performance in complex, unstructured real-world conditions requires strict process control and a tremendous amount of flight testing,” said Patrikar.
After validating ViSafe’s real-world performance through high-fidelity simulation, the team found that their system significantly reduces collision risk, even when operating under the strict size, weight, power and cost constraints typical of small uncrewed aircraft. These constraints make it impractical to rely on bulky, power-intensive sensors. ViSafe’s vision-only approach offers a lightweight, passive alternative that maintains strong safety performance, enabling broader deployment on agile, resource-constrained platforms.
“At closing speeds above 140 kilometers per hour, ViSafe must spot a 10 pixel speck half a kilometer away, crunch 4K video onboard in real time and steer clear of obstacles in less than ten seconds,” said RI Ph.D. student Nikhil Keetha. “These numbers show just how precise ViSafe’s capabilities are.”
