The Sound of Touch: On-body Touch and Gesture Sensing Based on Transdermal Ultrasound Propagation

Adiyan Mujibiya, Xiang Cao, Desney S. Tan, Dan Morris, Shwetak Patel, Jun Rekimoto


Recent work has shown that the body provides an interesting interaction platform. We propose a novel sensing technique based on transdermal low-frequency ultrasound propagation. This technique enables pressure-aware continuous touch sensing as well as arm-grasping hand gestures on the human body. We describe the phenomena we leverage as well as the system that produces ultrasound signals on one part of the body and measures this signal on another. The measured signal varies according to the measurement location, forming distinctive propagation profiles which are useful to infer on-body touch locations and on-body gestures. We also report on a series of experimental studies with 20 participants that characterize the signal, and show robust touch and gesture classification along the forearm.