Carnegie Mellon research shows AI-powered pilots could fly autonomous aircraft

New research from the Robotics Institute at Carnegie Mellon University shows that AI-powered pilots could enable the integration of autonomous aircraft into the airspace. Above is a simulation demonstrating the capabilities of an AI pilot. (Photo: The Robotics Institute, Carnegie Mellon University)

Researchers at Carnegie Mellon University have designed an AI (artificial intelligence) pilot that can navigate an aircraft through crowded airspace using visual flight rules (VFR). The AI pilot has been tested on flight simulators, and it detects other aircraft using a computer vision system and six cameras. It also offers the ability to communicate with air traffic controllers and pilots using its automatic speech recognition function.

There has not been sufficient research conducted on integrating autonomous vehicles into the airspace with manned aircraft, according to Jay Patrikar, a Ph.D. student at Carnegie Mellon’s Robotics Institute that worked on the AI pilot project.

“With air taxis, most have been focusing on the hardware—building the aircraft, making it go faster and farther,” Patrikar told Avionics International. “Nobody has talked enough about how these taxis would get integrated into the current airspace system.”

The AI pilot can operate an aircraft just like a human pilot, shared Dr. Jean Oh, associate research professor at the university’s Robotics Institute. She noted that larger airports usually have an air traffic control tower to optimize the schedule for aircraft taking off and landing.

At smaller airports, pilots rely on radio communications and visually observing other aircraft in the airspace to coordinate their actions. According to the Carnegie Melon team, most autopilot controls featured in commercial airliners and other aircraft are designed to operate under instrument flight rules (IFRs). That’s one reason why their team is focused on developing an AI pilot that can interact with other aircraft in the lower altitude VFR airspace where electric air taxis and drones operate.

“We ran a very simple scenario with two airplanes trying to land at the same airport, potentially creating conflict,” said Dr. Oh regarding the simulated flight tests. “They need to plan their trajectories so they avoid collisions safely.”

The AI predicts the future trajectories of other aircraft after observing their movements. “Based on those predicted trajectories, the AI planner chooses a safe action,” Dr. Oh told Avionics. “We compute all the possible cases and then choose the safest path.”

The researchers’ current approach focuses mostly on this trajectory-based intention prediction and planning. They are also working on a proof of concept for using natural language understanding. “We use communication to capture the other pilot’s intentions,” she noted. “When that information is available, the algorithm can use it to refine the planning.”

She remarked that there are limitations with this concept. “We can’t always assume that information will be ready. Communication is not always allowed or supported, and it can be very noisy input from the environment; there could be a lot of errors introduced by doing so.”

Dr. Oh has previously worked on robotic navigation systems for ground vehicles which helped to inform her work in developing an AI pilot. “A lot of the existing robotic technologies that are used for self-driving are developed in a static environment—the autonomous vehicle is the only moving agent in the environment and everything else is static,” she explained.

“The approaches have been developed to detect obstacles and avoid them. What hasn’t been fully addressed is there are many other vehicles and agents in the environment. In such dynamic environments, the research is still ongoing.”

There are also a lot of limitations in the current regulatory approach to allowing operations of unmanned aerial vehicles (UAVs) in the national airspace, according to Dr. Oh. “The Federal Aviation Administration and those currently designing the future of aviation are proposing a segregating approach,” she said. Essentially, they designate spaces for UAVs and other autonomous aircraft that are separate from those used by manned systems.

As the number of UAVs and drones grows over the next 10 years, issues will arise. Solutions currently in effect at airports are inefficient, says Dr. Oh. “At the airport, if there is one UAV landing or taking off, no one else can use that space. It is a very inefficient use of these expensive resources.”

She points out that the airspace will quickly become crowded, and not just for commercial use but in other areas as well. The approach of separating autonomous aircraft from other vehicles does not scale well. The AI pilot that the Carnegie Mellon team has developed could provide the ideal solution to these problems.

The AI pilot has not yet been tested on actual aircraft. “We are doing more user studies in the coming weeks to evaluate the AI pilot,” Dr. Oh remarked. “There are difficulties evaluating this integrated AI system, and any type of integrated AI for that matter—there is no good way to evaluate the system using one metric.”

The user studies, she explained, will involve human pilots flying in a simulated airspace with another aircraft controlled by either a second human pilot or the AI pilot. The research team will then ask the pilots how safe or comfortable they felt operating in each environment to evaluate the performance of the AI pilot. aviationtoday.com