If you’re working on your laptop in a crowded public space, like a coffee shop, an airport lounge, an industry conference or a co-working space, there’s one more thing you’ll need to worry about: Someone using a smartphone to eavesdrop on what you’re typing.
That’s according to data from a recent study published in the June edition of the Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies.
“I have had interest in the concept of keyboard usage and the information that it may convey for over 10 years,” told me Mitchell Thornton, a professor of electrical and computer engineering at Southern Methodist University (SMU) and one of the lead author of the study. “Additionally, my colleague, Dr. Larson and I have both had an interest in alternative uses of the sensor data produced by smartphone sensors. So this led us to the question to determine if there may potentially be some privacy concerns with regard to typing and smartphone sensor data.”
And the short answer is, Yes.
The researchers from SMU’s Darwin Deason Institute for Cybersecurity found that acoustic signals, or sound waves, produced when we type on a computer keyboard can be picked up by smartphone sensors including the microphone, the accelerometer, and the gyroscope.
While the microphone detects the sounds made by the keystrokes, the accelerometer and the gyroscope data was used to detect the faint vibrations that reverberate through a table when someone types.
To test their hypothesis, the SMU team built an iPhone app using artificial intelligence (AI) techniques and Swift, Apple’s open-source programming language, and were able to detect 41.8% of keystrokes and 27% of typed words correctly, even in a noisy environment—in a typical meeting scenario with people talking while others typed—and without user-specific training.
“We found that increasing the number of smartphones used causes overall accuracy to increase up to about 4 phones, but adding any more than 4 phones causes only minimal accuracy increases with our technique,” confirmed Thornton to me.
However, the security researchers found two major challenges for this attack. The first is that each type of keyboard is like a different drum with a specific sound. The second is that each table vibrates differently.
“The characteristics of the vibrations are affected differently depending upon the composition of the material of the table,” added Thornton.
Nevertheless, with more training on the different types of keyboards and tables, as well as a better understanding of the victim, such an AI-based software will have better detection rate regardless of the environment—noise, keyboard, table or typing style.
Atherton Research Insights
With this new discovery, you might want to think twice about the type of work you and your colleagues do in public spaces.
So far eavesdropping and spying on someone’s laptop was mostly limited to people peeking over-the-shoulder and looking at the screen, which can be easily avoided by installing a privacy filter on the laptop display.
Now, you will also need to install some kind of a shield over your keyboard to limit the acoustic sound emitted by keystrokes like a silicone keyboard skin which has the added benefit to also protect your keyboard from spills, dust, crumbs and key wear.