At present, finger input on touch screens is handled very
simplistically - essentially boiled down to an X/Y coordinate. However, human
fingers are remarkably sophisticated, both in their anatomy and motor
capabilities. TapSense is an enhancement to touch interaction that allows
conventional screens to identify how the finger is being used for input. This
is achieved by segmenting and classifying sounds resulting from a finger’s
impact. Our system can recognize different finger locations – including the
tip, pad, nail and knuckle – without the user having to wear any electronics.
This opens several new and powerful interaction opportunities for touch input,
especially in mobile devices, where input bandwidth is limited due to small
screens and fat fingers. For example, a knuckle tap could serve as a “right
click” for mobile device touch interaction, effectively doubling input
bandwidth.