Tired of Tiny Touchscreens? Smartwatches Can Now Track Your Finger in Mid-Air

University of Washington researchers demonstrate FingerIO.

First described by Leonardo Da Vinci and best known for its applications in naval warfare, sonar could be coming soon to your wrist.

FingerIO uses the speakers and microphones already built into smart devices to detect the position and motion of a finger using active sonar. A device with two microphones is required to track motions in two dimensions.

The technology has been tested with simple inputs such as flicks and scrolling motions, as well as with more complex inputs such as tracing shapes and letters in the air or on a surface near the device.

Tracking Fingers with Sonar

The technology uses 18-20 kHz Orthogonal Frequency Division Multiplexing (OFDM) signals, and related processing algorithms, to overcome the lack of synchronization between the microphone and speaker sampling rates on typical smart devices. This development is inspired by wireless communication systems, which often encounter similar synchronization problems.

Even with this, however, FingerIO is less precise than other tracking methods, such as cameras. The researchers who developed the technology report an accuracy between 0.8 - 1.2 cm, about the diameter of a finger.

On the other hand, FingerIO has the additional benefit of continuing to work even when fabric and other occlusions obstruct the direct line of sight between device and finger, unlike camera-based tracking technologies.

In user testing with off-the-shelf smartphones, the finger location computed by FingerIO (in green) was accurate to within 8mm of the actual motion recorded on a separate touchscreen (shown in black). (Image courtesy of the University of Washington.)

The team’s demo video shows significant noise when using FingerIO to draw a figure—yet the resulting “8” is still recognizable. The available real estate for gesture input—about 0.5m²x 0.25m² on either side of the device—is greatly improved over that of a smartwatch screen on its own, making this a particularly notable innovation for anyone with larger hands.

Making Smartwatches More Practical

Unlike smartphones, which have been steadily increasing in size over the years, the smartwatch faces the challenge of remaining small in order to stay practical. The prevalence of the touchscreen combined with the need for a small form factor has provided us with an unusual problem: an interactive display for which “interaction” and “display” interfere with one another.

FingerIO enables finger tracking via sonar. (Image courtesy of University of Washington.)

In order to interact with the Apple Watch or Moto 360, the user must often touch a screen not much larger than their finger. This not only limits the display of information, but also the number of interactive elements which can be displayed - too many elements require a higher precision than is accomplishable with a fingertip.

Although FingerIO is an inventive solution, there are other solutions to the smartwatch form-factor problem, including voice interfaces like Google Now, and compact keyboards such as the T9-style TouchOne app for Android Wear.

Further development of FingerIO could provide multi-input finger tracking, similar to multi-touch, as well as three-dimensional tracking with the addition of more microphones. In the future, similar technology could be used for life-size sketching of ideas, or for user interfaces designed for the blind.

The University of Washington is in the process of exploring options for commercializing the technology and releasing it to developers.

More information about the FingerIO project can be found here.