We spend so much time touching our devices — but what if they could touch us back? Apple’s recent leaps with 3D Touch technology show that this could soon be a reality. Feel a surface texture, communicate with braille, even play an instrument — the tactile capabilities of future technologies are set to transform our digital experiences.
In this article I will explore how developing technology is evolving the digital user experience with increasingly intuitive, expressive interfaces. I’ll also discuss the potential that 3D technology has to transform industries, whether it be in art, data analysis, gaming or e-commerce.
The evolution of digital metaphors
The digital representations used in computing interfaces communicate an action through the use of recognizable, real-world concepts. On a basic level, consider the trash icon on your desktop — dragging an object to the icon deletes the information, which is communicated through the concept of a trash bin.
For these metaphors to work, you need the right kind of hardware. For example, trying to interact with the desktop without a mouse would be tedious. Hardware dictates which interface allows the metaphors to make sense, and the interface metaphors dictate which tasks are easy or hard to perform. So, as hardware advances, we can adopt new metaphors and, in turn, shift and expand the scope of which tasks are easy to do on a computer.
With its introduction of the iPhone in 2007, which brought the multi-touch screen to the mass-consumer market, Apple has been a leader in this arena. Multi-touch let Apple create and execute new metaphors with the ability to stretch and squeeze with a two-finger pinch action, making navigating websites and maps faster and simpler. Here again, the right hardware enabled new metaphors, which in turn made 2D navigation easier, radically changing what is possible to do on a mobile device.
The force of a third dimension
The sensory technology of 3D Touch on Apple’s iPhone 6s is the latest advancement in this sphere to hit the market. This capability allows the device to recognize how hard you are pressing on the screen by detecting the minute bending of the touch surface caused by your finger. This is complemented by the Taptic Engine, which creates feedback through vibrations, described by Forbes as Apple’s “secret weapon.”
The physicality of touch has always been a fundamental part of how we engage with our world.
Together, the two provide a crucial combination — sensing and feedback — which allows for a new dimension of interaction through tactile pressure. This can be seen in the new iPhone “Peek and Pop” and “Quick Actions” functionalities. Using a light tap, “Peek and Pop” allows users to preview content, such as emails, without having to open them. By applying more pressure, a deeper “Pop” will open content in a new window. Using the same methodology, “Quick Actions” creates shortcuts for regular activities.
These actions display the first hints of a truly three-dimensional, layered space, and bring tantalizing new possibilities — some of which are already being tested today.
Microsoft researchers have explored this in 3D modelling through arranging objects by pushing them deeper into a 3D scene by pressing harder on the screen. Haptic feedback alerts the user when two objects bump. Microsoft has even created a 3D MRI, where pressing harder displays a deeper cross-section of the scan.
South Korean researchers have additionally used the force of a user’s touch to select the number of pages to turn when flipping through an e-book. This same technology has the power to let users control mobile devices without the need to look at the screen. As this technology matures, it could be revolutionary for everything, from automobile interfaces to accessible technology for the visually impaired.
All of these new metaphors and gestures let you do more on the screen, using less space and time for improved digital experiences. And it all comes from force sensing and simple tactile feedback.
What the future holds
Scientists are already exploring more complex tactile feedback, used to create far richer experiences. One approach is combining haptics with microfluidics (the manipulation of fluid) to change the physical properties and functionality of the touchscreen — presenting users with a sort of 3D interface. Phorm’s morphing touchscreen already uses this capability to provide pop-up “finger-guides” on a virtual keyboard; however, future possibilities could allow much more.
Disney has spent the last few years developing touchscreens that can change their surface friction in real time to create dynamic surface textures. The technology is strikingly effective, and the experience it creates could have profound effects for a number of industries. Imagine search results where texture signifies trustworthiness, or the impact on social media when people could feel the textures in their photos. And the future of gaming would be revolutionized through adding a third dimension of touch — not just looking for clues to a puzzle, but feeling for them, as well.
Creativity within the mobile device will soon begin to take a very new form. Sculpting clay figures on your iPhone screen, DJing with custom mix decks, testing a new pair of sunglasses before 3D printing them. These start to become possible with the right kind of tactile feedback. Quite a few remarkable projects in this vein have already been prototyped by MIT’s Tangible Media Group.
The most interesting applications are those we cannot even imagine today. How will mapping apps change when you can feel the contours of a landscape? How will data analysis change when analysts can gather information through touch as well as vision and sound? How will e-commerce change when you can feel the merchandise?
The physicality of touch has always been a fundamental part of how we engage with our world. It is both deeply instinctive, and rife with social meaning. The fact that the virtual, metaphoric worlds where we spend an increasing amount of our mobile time are largely devoid of touch is almost tragic. Restoring it will bring a new dimension to our digital interactions, expand the scope of what is possible on a computer and ultimately let us engage with the digital world in ways that are a little more human.