SideSwipe: Detecting In-air Gestures Around Mobile Devices Using Actual GSM Signal

Chen Zhao, Keyu Chen, Md Tanvir Islam Aumi, Shwetak Patel, Matthew S. Reynolds
PDF
The SideSwipe system. Our system leverages the unmodified GSM signal to enable in-air hand gestures both above and on the side of the phone.

Abstract

Current smartphone inputs are limited to physical buttons, touchscreens, cameras or built-in sensors. These approaches either require a dedicated surface or line-of-sight for interaction. We introduce SideSwipe, a novel system that enables in-air gestures both above and around a mobile device. Our system leverages the actual GSM signal to detect hand gestures around the device. We developed an algorithm to convert the discrete and bursty GSM pulses to a continuous wave that can be used for gesture recognition. Specifically, when a user waves their hand near the phone, the hand movement disturbs the signal propagation between the phone's transmitter and added receiving antennas. Our system captures this variation and uses it for gesture recognition. To evaluate our system, we conduct a study with 10 participants and present robust gesture recognition with an average accuracy of 87.2% across 14 hand gestures.