How ESA Astronauts Use AR and VR to Train for Space

Preparing astronauts for missions to the International Space Station (ISS) requires exploring the opportunities that different types of technology present as new and perhaps more efficient training tools. For astronauts training for space flight, there are many different segments to understand and integrate fully.

Apollo 11 captured the imagination of the world and put America’s space program squarely ahead of the Soviet program during the Cold War. It was perhaps the greatest feat of human engineering of all time.(Apollo 11.)

They have to learn about differing space station systems, operating robotic arms, space walking while waiting for a mission assignment from superiors. After an astronaut receives a mission assignment, specific training can take upwards of three years. The length varies depending on mission complexity. And you must speak Russian.


Practicing for launch is a huge part of training for any mission assignment, no matter the degree of mission complexity. Astronauts are completely strapped in and on their backs for hours prior to the launch. Seconds after the main engines light, solid rocket motors ignite, and the astronauts feel a massive jolt from underneath them. They are smooshed back into their seats with the force of twice their weight. Approximately two minutes pass and then the astronauts get a brief respite which disappears as they rise into the thinning atmosphere, burning off the majority of the fuel and pushing them down with about 2.5 g-force. Then astronauts go point-blank from the intense g-force to zero gravity, snapping right into weightlessness.

The training simulator can create similar shaking and noise to the actual launch, but it certainly can’t recreate the acceleration, speed and g-force astronauts feel as they go from zero to 17,500 mph in under nine minutes. A space shuttle carrying astronauts and cargo may dock with the ISS or the Russian MIR space station or to repair high-powered scientific instruments like the Hubble Space telescope.

Recently, a European Space Agency astronaut named Luca Parmitano posted a picture on Twitter showing himself and crew member Andrew Morgan (a NASA astronaut) using Magic Leap One to train for a new mission aboard the International Space Station. (Image courtesy of NASA.)

EVA stands for Extravehicular Activity, meaning astronauts will be wearing their spacesuits. The spacesuits are referred to as Extravehicular Mobility Units (EMUs). If space suits your interest, you are probably aware that NASA astronauts have trained for missions in underwater labs to simulate the effects of space walking.

NASA has a dedicated space at their Johnson Space Center in Houston called the Virtual Reality Laboratory, which was built in 1992. If you’re curious, you can actually download the flight software they use for EVA and robotics simulations, called Dynamic Onboard Ubiquitous Graphic (DOUG). DOUG along with other gear has been used for training astronauts since it’s was opened for all ISS missions and will continue to do so for future space explorations.

The Beyond Mission

In April of 2018, Luca Parmitano and his crew announced the name of their mission: Beyond. (Image courtesy of ESA.)
ESA astronaut Luca Parmitano is preparing for his Beyond mission. He’s using the HTC Vive to navigate through a virtual environment to learn about possible routes he will take when he’s spacewalking outside of the ISS. While he’s practicing these spacewalks, he’s also practicing how to simultaneously use equipment in microgravity. Parmitano flew up to the ISS in 2013, so he’s refamiliarizing himself with robotic operations and safety procedures while studying the new experiments the Beyond crew will be conducting over their six-month stay. (Image courtesy of the ESA.)

The research Parmitano is preparing to execute on the ISS is meant to deepen current knowledge on how to keep humans safer on lengthier exploration missions. Part of the Beyond mission will involve demonstrations by Parmitano to expand the operational procedures and technological capabilities necessary for robot-assisted humans to explore Mars and the moon from both their surfaces and from their orbits. According to Parmitano, “I see myself as a facilitator. My goal will be to put everybody in the condition to perform to the best of their capability.” Parmitano is training at NASA’s Johnson Space Center in Houston, the European Astronaut Centre in Cologne, and Star City, which is near Moscow.

The Magic Leap One EVA Simulation from DOUG for VR

The developer team for DOUG at NASA’s VRLab have successfully designed and built an application to help astronauts prepare and study the EVA runs they will have to perform in space on the mission.

Chief Engineer Evelyn Miralles has been posting pictures and singing the praises of the Magic Leap One augmented reality headset and the killer app built by software engineers who work on DOUG. (Image courtesy of NASA.)

Though it’s hard to get any information about the Magic Leap One application built for EVA training runs, one can extrapolate its features and perhaps a bit of the experience itself by understanding a bit more about DOUG.

Described as either a graphics engine or 3D viewing tool, DOUG gives astronauts access to mission-specific scene configuration databases. These are used in simulations at many NASA centers, which include Space Training Facility (SSTF), Space Station Remote Manipulator System (SSRMS) and Simplified Aid for EVA Rescue (SAFER).

Exploration applications are available through the graphics engine in a tool called Engineering DOUG Graphics for Exploration (EDGE). As we know at engineering.com, engineers love their acronyms (and acronyms where one letter stands for another acronym are even better).

The DOUG application helps prepare crews for their work, for review, situational awareness, EVA and robotics planning. It also helps them prepare during Neuro-Buoyancy Laboratory (NBL) training.

According to the user guide, DOUG works through EVA procedure animations for both EVA crew procedures and SSRMS positions for crew review and training onboard the ISS. By using the same software on the ground and onboard is to maximize the similarity of user interfaces and scene configurations throughout all systems.

The Magic Leap One for EVA must be similar to the existing one for VR headsets. So, after one downloads and runs the DOUG plugin package, a user selects VR Headset and the DOUG visuals appear on whatever Head Mounted Display (HMD)—Oculus, Vive and any of the Windows Mixed Reality headsets. (Image courtesy of NASA.)

Bottom Line

Training for EVA runs through DOUG’s simulations and procedure animation at NASA’s Virtual Reality Laboratory is as serious an application engineers could ask for as a good use case for the practicality of AR, VR and MR (XR).

Though Magic Leap hasn’t expounded on the use of its flagship product for training astronauts, the continued innovation by DOUG developers ensures that any chance space organizations like NASA and the ESA give astronauts like Luca Parmitano the best shot at completing their missions with as few hiccups as is humanly possible.