Just like a ray of light, a wireless signal bounces off of certain objects and surfaces. Different materials affect the wireless signal in different ways. Human bodies, therefore, cause particular changes when reflecting wireless signals. Using this property, MIT researchers from the Computer Science and Artificial Intelligence Laboratory (CSAIL) designed a wireless system that captures reflections off of humans and collects health and behavioral data.
The system, dubbed Marko, broadcasts radio-frequency (RF) signals that bounce off people in motion and return with specific changes. Specially-designed algorithms then analyze the reflected signals and associate them with specific people. In other words, Marko is capable of distinguishing individuals on the basis of wireless signals alone.
Moreover, the system monitors the movement of detected people within a defined digital space. Combining the collected movement information with other data can provide information about the individuals’ behavior and relations among people and with the environment. You can see how Marko works in the video below.
For testing and demonstration purposes, the MIT team monitored a patient diagnosed with dementia who would often become agitated for unknown reasons. This was performed in one assisted living facility, with permission from the patient’s family and caregivers. The researchers concluded: “For over a month, Marko has measured the patient’s increased pacing between areas of their unit — a known sign of agitation. By matching the increased pacing with the visitor log, they determined the patient was agitated more during the days following family visits. This shows Marko can provide a new, passive way to track functional health profiles of patients at home.”
The system is deployed in a targeted space and transmits RF signals. The rebounded signals create a map cut into vertical and horizontal frames that determine where individuals (represented as bright blobs) are in a 3D space. Horizontal frames define the person’s general position, while vertical frames determine their characteristics, such as height and build.
The system generates short user trajectories, called “tracklets”, by analyzing the RF frames (30 per second) when the person moves. To separate the reflections of given individuals and process their movements, the researchers used what is called a “convolutional neural network”, a machine-learning model usually used for image recognition. The team implemented a filter, represented as small circles around each person’s body. This filters out signals outside of the circle as it moves. The system combines the aforementioned information and associates specific RF reflections with specific individuals.
Before putting Marko to work, the system needs to collect information about targeted individuals. Users of interest need to wear low-powered accelerometer sensors which tag the reflected RF signals with their corresponding identities. As a first step, the system generates individuals tracklets. Then, an algorithm associates certain acceleration features with motion features. When the users walk the measured acceleration oscillates with each step, then becoming a flat line when they stop. The system determines the best congruence between the acceleration data and tracklet, and tags that tracklet as the user.
In this way, the system learns which reflected signals correspond to specific identities. After this initial data collecting, there is no need to wear the sensors again. The system is quite efficient and able to tag the identities of individuals with up to 95 percent accuracy.
Probably the most important potential application is in healthcare facilities where Marko can passively monitor how patients interact with family, caregivers, and nurses, whether they receive medications on time, etc. It could also be used as a new approach to collecting data about family dynamics, daily schedules, sleeping habits, etc. Marko is a more comfortable and affordable solution to cameras, in that it provides greater privacy and simply focused on behavioral patterns, motion, sleep, and interaction in the specific area and time.
Despite these privacy benefits, the authors do note “the importance of ensuring the technology does not get misused to infringe on privacy.” Because MIT CSAIL was founded on the basis of a Department of Defense (DoD) project and continues to use DoD funding, there are obviously military and counterinsurgency applications for which Marko could be deployed. In these cases, it is essential for civilians to understand how their privacy might be compromised.
This interesting innovation has been published in a paper for the CHI Conference on Human Factors in Computing Systems Proceedings.