What is spatial intelligence? No, it’s not a measure of how many constellations you can pinpoint in the night sky. That might impress your date, but it won’t impress George Shaw, founder and CEO of Pathr. Shaw has spent his entire career tracking much trickier targets: human beings.

“Pathr is a spatial intelligence company. What that means is we understand how people move through physical spaces,” he explained.

What Good is Spatial Intelligence?

There are many reasons to study how people move through physical spaces, and Shaw has dipped his toes in just about all of them. He kicked off his career at MIT’s Human Speechome Project, a study of language acquisition that monitored a child for three years via continuous audio and video recordings. Shaw used the video footage to track the movement of people throughout the home and connect it to the child’s language development.

Shaw expanded that work into the commercial sector with Bank of America and Best Buy, and then he left MIT to work for a company called Retail Next, which provides analytics to brick-and-mortar stores. After that, Shaw became the head of R&D at AltSchool, a start-up private school that sought to tie student location data to learning outcomes.

If language studies, retail stores and private schools don’t do it for you, however, Shaw’s next venture ought to get you excited about spatial intelligence. He moved on to a company called Second Spectrum, which provides spatial tracking and analytics for major sports leagues including the NBA, Premier League and MLS. Do you know how many metres Kawhi dribbled last season? Because Shaw and his team do.

“Second Spectrum uses the location of players on the basketball court to understand the outcomes that a coach would care about. It’s currently being used by every NBA team,” Shaw told us.

After Second Spectrum, Shaw went back to the retail game as a data strategist for Target, and then he moved on to Intel to help build a platform for computer vision at the edge. “Location understanding is something I'm really familiar with,” he understated.

That familiarity spurred Shaw to start his own spatial intelligence company: Pathr.

“Second Spectrum was the inspiration. The way they were using machine learning on location to understand basketball, I thought, was applicable across lots of industries. And so, I launched Pathr to pursue that,” he explained.

On a New Pathr

Say you’re in charge of a manufacturing facility. Widget production is going great, but you have another problem: keeping your employees healthy. There’s a new coronavirus spreading around, and you want to de-petri-dish your shop floor as much as possible. You don’t want to end up like the meat processing plants where thousands of shoulder-to-shoulder workers were diagnosed with COVID-19.

So, what to do? To keep your employees safe, you need to understand how they move through your space. You need spatial intelligence. That’s where Pathr comes in.

Pathr takes an environment—a factory, an office or a retail store, it could be anything—and models it in 3D. They use Unity, a real-time 3D development platform.

“We build up the environment in Unity, except we're not concerned with the visuals at all,” Shaw explained. “We care a lot about the physics, but not the visuals. Our environments are very ugly, but they generate the right data and the physics are the way we want them to be.”

The 3D model is step one. The next step is to build a behavioural model of the people in the environment. To do that, Pathr needs data—any data will do. The company uses existing sources including surveillance cameras, IoT sensors, machine signals, badge swipes and anything else that can be used to understand the space.

“That's one of the differentiators that we have, that we can generate location data from the signals that we get from all these different devices,” Shaw said. “All that being said, our two favorites and I think the most ubiquitous, are existing surveillance cameras and simulation.”

Yes, simulation. Once the behavioral model is set up, Pathr can simulate people moving in the Unity environment as though they are NPCs in an old-school video game. That turns out to be a valuable source of data itself, and one particularly well-suited to our current circumstances.

“During the early days of the pandemic, there was no way to collect data. Retail stores were closed, factories were closed, offices were closed. We couldn't get real data,” Shaw recalled.

Unity Simulation

Back to your manufacturing facility. You’ve enlisted Pathr, and they’ve created a 3D model of the factory floor in Unity. Using surveillance cameras, machine signals and other sensors, they’ve created a behavioural model for each different job role at the plant. Now all that’s left to do is turn on Unity and run a whole bunch of simulations.

That last part is now a little easier for Pathr, thanks to a new service called Unity Simulation. Unity Simulation is a managed cloud service for running a series of parameterized Unity builds. Before Unity Simulation, Pathr had cobbled together their own simulation backend.

“We were kind of putting a square peg in a round hole initially,” Shaw explained. “We had built this simulation framework, but we weren’t using Unity the way most people were using it. And now, with Unity Simulation, we're perfectly aligned. We are in the evaluation stage, but we really love the platform.”

Alright, so Pathr is running the simulation of your manufacturing facility. Unity Simulation is creating synthetic data up in the cloud, and you’re sitting back eagerly waiting for the results. How long do you have to wait?

“It's getting faster and faster,” Shaw proclaimed. “That's one of the things that we're really excited about. The Unity team and our engineering team have set this goal of being able to generate a year of data in a few hours. We're not quite there yet, but we're much faster than real time, which is already really powerful.”

So, you wait a few hours—call it a day for good measure—and there it is. A year’s worth of data on how your employees are moving through your facility—and more importantly, insight into what you can do to keep them social distancing properly.

This is a real example of one of Pathr’s manufacturing customers, and the facility ended up changing the timing of their employees’ shifts, the number and organization of break rooms and locker rooms and the frequency of break room visits, among other modifications.

“We built this massive simulation, many billions of base data points, and then we ran our social distance analytics on that location dataset and gave them results that they actually used to change the way they manage their shifts,” Shaw said.

No Bulldozers Required

Spatial intelligence is useful wherever there’s space, and Pathr applies its technology across industries. Aside from manufacturing facilities, Pathr has worked with several office buildings to optimize workspaces for social distancing and get everyone back to work safely.

But spatial intelligence isn’t just for COVID. In the retail space, Pathr’s technology helps companies understand their layouts and how to improve them.

For example, Shaw told us about a shopping mall with which Pathr has worked extensively. For that project, Pathr used nothing more than existing surveillance cameras to gather the real-world data (though Pathr did install a server to process the many feeds). That data was used to develop a granular behavioural model that accurately reflects real mall shoppers.

“We have some confidence that our agents are moving the way real shoppers move, that traffic densities are what they would be in the real mall,” Shaw asserted. “It's basically a digital twin of the mall, and we can take and modify that digital twin.”

With this twin, Pathr can help the mall determine the effect of any possible layout change.

“What happens if you add an entrance? What happens if you expand the food court? What happens if you close down a hallway? All these kinds of changes are really difficult to do in real life. You wouldn't go and bulldoze half your mall just to try it out. But in the digital world with the digital twin, you can do that,” Shaw asserted.

The Future of Spatial Intelligence

Pathr’s work is just getting started, and Shaw envisions many ways that spatial intelligence could become even more intelligent. For example, while his team manually implemented layout changes to the shopping mall to simulate their effect, in the future these changes could be automated with optimization algorithms like those used in generative design.

Another possible future for spatial intelligence is one that requires less and less real-world data.

“I think we'll never get to the point where we don't want real-world data, but I do think the simulations that we run and the synthetic data that we generate will get more and more realistic,” Shaw predicted.

To learn more about Unity and to try Unity Simulation for free, click here.