Drones are becoming more sophisticated, with better controls and longer battery life, and are seeing use for everything from hobbyist drone pilots, photographers and hikers to the military or search and rescue. While these drones often have flight stabilizers or GPS to help navigate, flying drones using joystick-style controllers is still difficult to learn and even harder to master.
But what if there was another way to fly a drone that was more engaging to the operator, easier to learn, and a lot more fun?
Instead of using your thumbs—and hoping you can keep things coordinated—imagine piloting a drone using movements of your torso, leaving your head free to move and look around, much like a bird. This is what a team of researchers has been developing at Ecole Polytechnique Fédérale de Lausanne (EPFL). Their research demonstrates that using the torso to pilot flying machines is much more immersive and intuitive for the operator—and more effective—than using the long-established joystick.
"Our aim was to design a control method which would be easy to learn and therefore require less mental focus from the users so that they can focus on more important issues, like search and rescue," says Jenifer Miehlbradt of EPFL's Translational Neuroengineering Laboratory. "Using your torso really gives you the feeling that you are actually flying. Joysticks, on the other hand, are of simple design but mastering their use to precisely control distant objects can be challenging."
No More Joysticks
While developing their system, the team approached the pilot problem by first determining what movements were the most natural and intuitive when attempting to use their body to pilot a flying object—in this case a drone.
First, the research team monitored the body movements of 17 individuals by placing 19 markers all over the upper body. This also allowed them to monitor muscular activity. Participants followed the actions of a virtual drone through simulated landscapes that they passed through while wearing virtual reality goggles.
Common motion patterns emerged, which the team used to establish torso-related strategies for piloting drones. In the end, they determined that only 4 markers, located on the torso, were needed to successfully and effectively pilot both flight simulators and real drones through a circuit of obstacles.
Comparing torso strategies to joystick control in 39 individuals, the team found that participants using the torso drone control method outperformed joystick control in both precision and reliability, and required only minimal training and practice sessions.
"Data analysis allowed us to develop a very simple and intuitive approach which could also be used with other populations, machines and operations," says Silvestro Micera from the biomedical engineering department at Scuola Sant'Anna in Italy in Biomedical Engineering. "The approach significantly improves the teleoperation of robots with non-human mechanical attributes."
So far, the development of the torso control strategies provides unique and highly immersive piloting strategy with a focus on characterizing the relevant torso parameters. This leaves the head, limbs, hands and feet all free to perform other actions, although their proof-of-concept system still requires body markers and external motion detectors in order to work.
Next steps include making the torso strategy into a completely wearable system for piloting flying objects. The range of potential applications is huge—everything from flight simulators to piloting drones and maybe even flying the airplanes of the future.
EPFL has already started development in this direction, working on a garment-like system that implements the torso strategy into drone control without requiring external motion detectors.
Fly Like an Eagle
It’s been dubbed the Fly Jacket, and is the first implementation of the torso strategy for piloting flying objects.
The goal—at which they are succeeding so far—is to give drone pilots an unprecedented level of immersion. The Fly Jacket development is being led by Carine Rognon, a researcher at EPFL’s Laboratory of Intelligent Systems.
Rognon designed a very lightweight garment that translates the torso strategy described in the Translational Neuroengineering Laboratory’s research into drone control commands without requiring external motion detectors. This allows pilots to control their drone intuitively using their body movements – such as leaning forward and backward and pivoting their upper body – rather than using a hand-held device like a joystick or other controller.
“The Fly Jacket is made of two layers, a more rigid layer in order to anchor sensors and transmit forces, and a softer layer in order to allow the person to move freely,” said Rognon.
An embedded motion sensor developed by EPFL’s Laboratory of Embedded Systems makes the Fly Jacket drone control extremely responsive. The fly jacket also has an arm support system that helps prevents fatigue during longer operation times. Together with the use of first-person view (FPV) goggles connected to an on-board camera, this allows pilots to naturally navigate the skies while keeping their arms spread out like wings.
“The Fly Jacket not only produces an immersive and intuitive flight control experience, but also frees the human hands for other tasks. For example, we have shown that humans could wear data gloves to give additional commands to the drone, like for take-off and landing, or to indicate points of interest seen from the drone’s perspective that would immediately appear on a map. This could be very useful for fire fighters or rescuers to quickly and precisely identify locations where help is needed,” says Floreano.
Rognon added, “We would like to offer a kit, with the jacket and the drone. The rescuer can wear it, and do more intuitive body movement than using joysticks, and they can also do other things because their hands are free.”
For more on innovative ways to use and control drones, check out these stories: