Jinha Lee wants to change the way we interact with digital screens.
In his TED Talk, Reach into the computer and grab a pixel, Dr. Lee shows some current projects and discusses future possibilities. The talk begins with discussing the boundaries between the user and the screen, and throughout the talk the boundary gets smaller until it no longer exists.
Using a light pen Lee tells how the user can now draw into the screen, creating a structure inside the plane of the interface. Next he shows a see-through desktop that allows the user to exist inside the screen. When files have physical locations in 3d space then accessing those files is like pulling a book off of your shelf.
http://leejinha.com/See-Through-3D-Desktop
The next step is having the digital information manifest itself outside the screen. The talk shows a wrist trying on a watch via smartphone screen. Now we have the possibility to virtually try on items before buying and shipping them.
To create a physical pixel Jinha created a spherical magnet that cancels gravity and uses sensors to exist in space. The object can now record human motion and exist in more than one place, and transmit the motion pattern to another location. A highlight of this system is two sensor-hacked ping pong paddles hitting a virtual ball across the interface.
Many applications exist now and more are being developed with Lee and the MIT Media Lab. Wristwatches for the blind, multicolored pixelated blocks, and augmented street reality are some highlights from Lee's incredibly dense website.