Deep Learning Teaches Drones to Survive Any Weather

Science News -- May 7, 2022: Drones, or autonomous flying vehicles, will need to learn to navigate real-world weather and wind conditions in order to be genuinely useful. Drones are now either flown in controlled environments with no wind or by people using remote controls. 

Drones have been taught to fly in formation in open skies, although these flights are normally carried out under ideal conditions.

However, in order for drones to do important but mundane duties like delivering parcels or airlifting injured drivers from a traffic collision, they must be able to adapt to wind conditions in real time — in other words, they must be able to roll with the punches, meteorologically speaking.

To meet this difficulty, a group of Caltech engineers created Neural-Fly, a deep-learning technology that can assist drones cope with new and unfamiliar wind conditions in real time by just changing a few essential parameters.

A paper published in Science Robotics on May 4 describes Neural-Fly. Soon-Jo Chung, a Bren Professor of Aerospace and Control and Dynamical Systems and a Jet Propulsion Laboratory Research Scientist, is the corresponding author. Michael O'Connell (MS '18) and Guanya Shi, both Caltech graduate students, are co-first authors.

The Real Weather Wind Tunnel, a bespoke 10-foot-by-10-foot array of more than 1,200 tiny computer-controlled fans that allows engineers to mimic everything from a moderate gust to a storm, was used to test Neural-Fly at Caltech's Center for Autonomous Systems and Technologies (CAST).

"The problem is that the direct and specific effects of varied wind conditions on aircraft dynamics, performance, and stability cannot be effectively captured by a simple mathematical model," explains Chung. "Rather than attempting to qualify and quantify every effect of the turbulent and unpredictable wind conditions that we frequently encounter in air travel, we use a hybrid approach of deep learning and adaptive control that allows the aircraft to learn from previous experiences and adapt to new conditions on the fly while maintaining stability and robustness."

O'Connell continues: "We have a variety of fluid mechanics models, but getting the proper model fidelity and adjusting it for each vehicle, wind situation, and operating mode is difficult. Existing machine learning algorithms, on the other hand, require massive quantities of data to train and do not match state-of-the-art flight performance achieved with traditional physics-based methods. Furthermore, real-time adaptation of a complete deep neural network is a large, if not currently impossible, undertaking."

The researchers claim that Neural-Fly overcomes these obstacles by employing a "separation technique," in which just a few parameters of the neural network must be changed in real time.

"We achieve this with our unique meta-learning technique," Shi explains, "which pre-trains the neural network so that only these critical parameters need to be updated to properly capture the changing environment."

Autonomous quadrotor drones equipped with Neural-Fly learn how to respond to severe winds in as little as 12 minutes of flying data, and their performance improves dramatically (as measured by their ability to precisely follow a flight path). When compared to current state-of-the-art drones equipped with similar adaptive control algorithms that identify and respond to aerodynamic impacts but without deep neural networks, the error rate following that flight path is roughly 2.5 to 4 times lower.

Neural-Fly is based on prior systems known as Neural-Lander and Neural-Swarm, and was created in partnership with Yisong Yue, Professor of Computing and Mathematical Sciences, and Anima Anandkumar, Bren Professor of Computing and Mathematical Sciences, at Caltech. Neural-Lander also used a deep-learning method to track the drone's position and speed as it landed, modifying its landing trajectory and rotor speed to compensate for the rotors' ground backwash and achieve the smoothest possible landing; Neural-Swarm taught drones to fly autonomously in close proximity to one another.

Despite the fact that landing appears to be more difficult than flying, Neural-Fly, unlike previous systems, can learn in real time. As a result, it can react to changes in wind on the fly and does not require post-processing. Outside of the CAST facility, Neural-Fly performed just as well in flight tests as it did in the wind tunnel. Furthermore, the team has demonstrated that flight data collected by one drone can be transferred to another drone, forming a knowledge pool for autonomous vehicles.

Test drones were tasked with flying in a pre-described figure-eight pattern while being bombarded with winds up to 12.1 meters per second — around 27 miles per hour, or a six on the Beaufort scale of wind speeds — at the CAST Real Weather Wind Tunnel. This is defined as a "strong breeze" that makes using an umbrella difficult. It's barely below a "moderate gale," which makes it difficult to move and causes entire trees to sway. This wind speed is twice as fast as the drone experienced during neural network training, implying that Neural-Fly might extrapolate and generalize well to unknown and severe weather conditions.

The drones were outfitted with a typical off-the-shelf flight control computer utilized by the drone research and enthusiast communities. Neural-Fly was built into an onboard Raspberry Pi 4 computer, which is about the size of a credit card and costs roughly $20.

Journal Reference:

Michael O’Connell, Guanya Shi, Xichen Shi, Kamyar Azizzadenesheli, Anima Anandkumar, Yisong Yue, Soon-Jo Chung. Neural-Fly enables rapid learning for agile flight in strong winds. Science Robotics, 2022; 7 (66) DOI: 10.1126/scirobotics.abm6597

Cite This Page:

MLA
APA
Chicago
California Institute of Technology. "Rapid adaptation of deep learning teaches drones to survive any weather." ScienceDaily. ScienceDaily, 5 May 2022. .
 
 Wnctimes by Marjorie Farrington