A new prototype for a search-and-rescue drone successfully located people in dense forests about 90% of the time in 17 field tests across different forest types and seasons. The design, presented in the June 23 issue of Science Robotics, integrates thermal imaging, machine learning, and a relatively new optical algorithm to help the drone see lost persons through the trees.
"Our main motivation is to provide a potentially life-saving technology which is more flexible, faster, and possibly more efficient than manned helicopters," said corresponding author Oliver Bimber, professor of computer graphics at Johannes Kepler University Linz in Austria. "Drones can also fly in weather conditions where helicopters can't, and AI-based person classification together with the AOS [algorithm] is potentially much more reliable than an observing pilot."
The new system could propel the use of drones in a variety of response operations, such as wildfire rescue or fugitive pursuit. "It is also important to note that the presented technology has substantial potential for other applications within response robotics and beyond," notes Andreas Birk in a related Focus article, citing wildlife monitoring or military operations as possible examples.
Robots offer many advantages over humans in search and rescue. They can operate from a safe distance, cover a wide range, and are packed with sensors to detect signs of life and recognize threats more accurately. They are also faster and more cost-effective.
But even state-of-the-art drones used for search and rescue operations remain remotely operated by a human, who visually inspects the drones' field of view. Relying on the limited scope of human vision, drones cannot easily see underneath a leafy canopy or through thick fog, for example.
Bimber and his team worked to overcome this limitation by employing an airborne optical sectioning (AOS) algorithm in combination with thermal imaging. The use of thermal cameras to visualize warm bodies is a standard technique for response operations, but AOS is relatively new to search and rescue. It works by computationally defocusing objects that block the ground — such as forest cover — in the drone's field of view, so that targets on the ground become clearly visible.
After clearing away the flora, the drone can more easily detect human bodies, aglow in the sights of thermal imaging. It then identifies its target person using machine vision, which is previously trained to recognize the person of interest based on various classifications.
"In our previous work, the flight path was predefined, and all the processing was done on the ground, after the flight," said Bimber. "Now, all the processing is done in real-time on the drone, allowing the drone to make its own decisions on how and where to fly — based on what it "sees" during flight — with the goal to find a person as quickly and as reliably as possible."
In 17 field experiments conducted over conifer, broadleaf, and mixed forests that vary in light levels, temperature, and seasonality, the AOS-equipped drone found 38 out of 42 hidden persons in free flying conditions. For experiments with predefined flight paths, the drone demonstrated an average success rate of 86%.
By processing all of its data on board, the drone could operate effectively even in areas without stable network coverage.
The researchers aim to further improve the short battery life of the drone, which used here restricted flying times to 15 to 20 minutes. They hope to apply their AOS to more long-lasting drone prototypes to extend total flight time and reduce search time. They are also thinking of ways to prepare their system for collaboration with human emergency responders.
Bimber and his team have already received an array of requests from various research teams that wish to employ their AOS.
"We did archaeology and wildlife observation in previous studies, but tracking and counting of wildlife is also a very common request we get — from red deer and bison to Bigfoot even," said Bimber. Wildfire hotspot detection is also of interest, he added.
"What excites us most is that AOS has so many applications," said Bimber. "That is why we decided to make all our source code and data available to everybody for free for non-commercial usage."
[Credit for associated image: Johannes Kepler University Linz]