James Patten wants to change the way we interact with our computers. He built an interactive display at the Museum of Science and Industry in Chicago, enabling users to grab elements off of the periodic table and combine them to create chemical reactions. Due to the heavy bond created between user and machine when his interfaces were used, he decided to create new ways to control computers.
In his TED Talk, The best computer interface? Maybe... your hands, Patten begins by explaining some of his favorite projects and then focuses on his work with control theory. He discovered that using physical objects specifically for the task of learning about chemical reactions, and that our brains are wired to manipulate tangible objects.
http://www.ted.com/talks/james_patten_the_best_computer_interface_maybe_your_hands#t-355839
His first project was an electromagnetic array at the MIT Media Lab. The magnets allowed movement of objects but were very expensive and very heavy. James began to build several prototypes, many of them incorporating omniwheels. The goal was to move objects that existed only in the computer, and create new interactions with the objects.
An amazing example from the talk shows three robots buzzing on top of a large screen. The application is for video editing and the controls for the editing functions are tied to the different bots. Users are given the functionality and feel of physical dials by manipulating the minibots and then controlling the software functions.
Patten's project are huge in scope, and the applications range from medicine to disaster response to gaming and video chat. The only drawback I see is that the controllers have to be built into the application from the very beginning to be effective. Imagining a future computer as a coffee table with robots rolling through projected space is very exciting.
http://www.ted.com/talks/james_patten_the_best_computer_interface_maybe_your_hands#t-355839