Researchers develop system that lets smartphones interact with objects using electromagnetic sensing

Carnegie Mellon University
INSTANT CONNECTION: CMU’s new sensing technology enables appliance recognition and interaction

Researchers from Carnegie Mellon University’s Future Interfaces Group have developed a working concept that enables smartphones to interact with smart home appliances using electromagnetic emission sensing technology.

“Most closely related to our technical approach are methods that perform recognition by sensing the electromagnetic emissions of electrical, electromechanical and electronic devices,” the researchers explain. “We have presented a novel system that enables users to simply tap their smartphone to an appliance to interact with it.

“To achieve this, we developed a novel hardware sensing configuration and combined it with efficient and accurate real-time classification to create a self-contained prototype. Our proof-of-concept smartphone recognises physical contact with uninstrumented appliances — this is done by sending the electromagnetic signature emanating from electronic devices and recognising the device in real-time.”

“Once recognised, appliance specific functionality can be launched automatically,” the researchers add. “We explore two approaches to ease the interaction bottleneck between users and the ecosystem of smartwatch appliances.

“The most straightforward option is to automatically launch manufacturer’s applications instantly upon contact with an associated appliance, without requiring users to find or remember which app corresponds to which device.

“For example, touching a smartphone to a thermostat would launch the thermostat’s configuration app. In this case, the currently running app on the phone is swapped out for a new full screen app.

“Alternatively, the phone can expose what we call contextual charms — small widgets that allow the currently running smartphone application to perform actions relevant to the touched appliance. For example, if reading a PDF, touching the phone to a printer will reveal an on-screen print button.”

A video gives an overview of how the concept works:

“As appliances are typically long-lived and rarely replaced, the adoption rate of smart appliances is expected to be slow — few, if any appliances in the average home today are e.g. NFC-enabled,” the researchers say.

“Thus, our technical approach requires no modification or instrumentation of appliances and can therefore work out of the box with already deployed devices.”