Mixed reality is making headlines all over the world, mostly in the media and entertainment space. After a recent conversation with an inside source at Microsoft Hololens division, I wanted to check in and see how the military uses augmented reality for training or even possibly on the battlefield.
In this post, mixed reality for militaries and law enforcement will be explored to try and find connections between commercial entities with military contracts and the current commercial, industrial sectors where mixed reality and augmented reality are being promoted as productivity enhancers and new mediums for media and entertainment. There are many industries where great engineering helps keep people safe. It’s also relevant to keep in the back of your mind the notion that weapons engineering is inherently political, because it involves innovation for the sake of waging war and keeping your enemies intimidated.
Microsoft and Osterhaut Design Group
When Microsoft bought the Osterhaut Design Group (ODG) patents to begin its HoloLens project in 2015, ODG was just a small company doing business with the U.S. military. But what was it doing exactly?
ODG is working with the Office of Naval Research (ONR) and TechSolutions to give U.S. Marine signals intelligence (SIGINT) specialists an augmented environment for the battlefield by designing ODG R series augmented-reality glasses to create digital overlays of real-time information. The partnership revolves around designing and modifying the X-6 system.
It has 1280 x 720 stereoscopic optics to provide a virtual display and built-in 802.11 and 802.16 communications for transferring and receiving data. There is a 1.5 GHz CPU embedded for processing and a rechargeable lithium ion pack. The glasses only weigh 4.5 ounces, which is not too bad compared to many commercial headsets.
ODG and TechSolutions modified the X-6 glasses to have a toggling weapons-mounted interface that allows Marines to take their positions and fire their weapons accurately. The OS is Android, which allows the Marines to test out and deploy new applications easily.
Other applications include directional markers, maps of the surrounding environment, friendly force tracking and sending out alerts to different groups of soldiers.
SIGINT instructor and staff sergeant Nicholas Lannan worked together with U.S. Marine modeling and simulation expert Major Christian Fitzpatrick to create a hands-free, Android-based augmented-reality device to simply replace the need to check his government-issued smartphone. The smartphone coupled with the antenna systems he was carrying made him less inconspicuous than is desirable in a combat situation.
Throughout this coordinated effort to create battle-ready augmented-reality glasses, a few problems kept coming up, namely that the headset was very fragile, and very expensive at around USD$20,000.
The R-6 glasses, which were commercially available for around USD$5,000 were used in a battlefield simulation near North Carolina at Camp Lejeune. A program built by MIT simulated the digital presence of thousands of civilians networked with a criminal organization seeking advanced weapons to attack naval vessels. The goal was for the Marines to use the ODG glasses to filter out details of the simulated deal and disrupt it. Understanding how enemies use a variety of digital communication methods is tantamount to predicting their tactical decisions.
Recent HoloLens Activity From the ONR
In pursuing their research about how helpful augmented reality could be for small-unit leaders, a group known as “The Spartans,” from the 2nd Battalion, 6th Marines held a technology training week at Camp Lejeune.
Also dubbed “Spartan Week,” the event is an opportunity to use ONR technologies, including an Interactive Tactical Decision Game (I-TDG) with a HoloLens, as well as a UAV (Unmanned Aerial Vehicle) survey system for modeling the ground from the sky. A person known as an augmented immersive team trainer uses the Web-based I-TDG to conduct simulation exercises and games that test tactical prowess.
This capability allows small-unit leaders to practice different battlefield situations and view the results of their simulations and games, like the way professional athletes will watch footage of practices or games to improve their skills.
Per senior research psychologist Natalie Steinhauser, who traveled from the Naval Air Warfare Center Training Systems Division in Florida, “Small-unit leaders are tasked with making big mission decisions in an extremely short time window. Decisions not only impact the success or failure of a mission, they affect life and death. With technologies like the I-TDG, Marines can perform simulated missions in a safe classroom environment, carry out multiple missions and even use the I-TDG as an after-action review tool.”
After a conversation with an inside source at Microsoft’s HoloLens division, it’s off the mark to think Microsoft wants to convey anything about HoloLens other than that the company is now keepers of a universal OS for mixed-reality devices that provides developers with a software platform to create dynamic content for enterprise and industrial customers.
Microsoft has nothing against consumer media like gaming of course, but for now, it’s going to step back and let the new mixed-reality OS hopefully take off as the software of choice for many new augmented-reality hardware devices as well as virtual-reality devices that will be manufactured by companies like Lenovo, Dell, HP and Asus. The idea is to unify the software so that manufacturers can make augmented-reality computing devices that run on Windows. Microsoft’s rebranding of Windows Holographic OS as the new mixed-reality OS shows that it believes the term is now an industry standard.
We’ll see how augmented reality continues to be used for military purposes, but for now, it doesn’t seem like the new wave of augmented-reality devices such as the R series from ODG or the HoloLens from Microsoft is going to the battlefield just yet.