PROJECT

SymmetriSense: Mobile Platform for Enabling Near-surface Interactivity on Everyday Glossy Surfaces using a Single Commodity Smartphone

We developed SymmetriSense, a technology that enables near-surface 3-D fingertip localization above arbitrary glossy surfaces, using a single commodity camera such as one from a smartphone. The state-of-the-art requires dedicated devices such as depth cameras or stereoscopic cameras which are still not as ubiquitous as a regular camera built in every smartphone. SymmetriSense localizes a user’s fingertip by using only a smartphone, which is ubiquitously available. To address the challenge using a single camera, we proposed a novel technique utilizing the fingertip's natural reflection and the principle of reflection symmetry. SymmetriSense achieves sub-centimeter 3-D localization accuracy in most cases, even as environmental conditions change.

*This work was done when I was an intern at IBM Research Austin.

*This work was published in CHI 2016. [ACM DL][Video]

---

PowerForecaster: Predicting Smartphone Power Impact of Continuous Sensing Applications at Pre-installation Time and Developing Context-aware Battery Management Advisor

Today’s smartphone app markets miss a key piece of information, power consumption of apps. This causes a severe problem for continuous sensing apps as they consume significant power without users’ awareness. Users have no choice but to repeatedly install one app after another and experience their power use. To break such an exhaustive cycle, we proposed PowerForecaster, a system that provides users with power use of sensing apps at pre-installation time. Such advanced power estimation is extremely challenging since the power cost of a sensing app largely varies with users’ physical activities and phone use patterns. PowerForecaster adopts a novel power emulator that emulates the power use of a sensing app while reproducing users’ physical activities and phone use patterns, achieving accurate, personalized power estimation. Our experiments with three commercial apps and two research prototypes show that PowerForecaster achieves 93.4% accuracy under 20 use cases. Also, we optimize the system to accelerate emulation speed and reduce overheads, and show the effectiveness of such optimization techniques.

While studying the power consumption characteristics of continuous sensing apps, we found that many continuous sensing apps not only consume battery in the background continuously, but also their battery uses vary depending on users’ contexts. Without understanding such characteristics, users may perceive growing disparities between their estimation of the near-future battery consumption and the actual outcomes. To help users effectively manage their battery, we developed Sandra, a novel context-aware smartphone battery information advisor. *These works were published in SenSys 2015 [ACM DL][Paper] and UbiComp 2015 [ACM DL][Paper].

---

Exploring Current Practices for Battery Use and Management of Smartwatches

As an emerging wearable device, a number of commercial smartwatches have been released and widely used. While many people have concerns about the battery life of a smartwatch, there is no systematic study for the main usage of a smartwatch, its battery life, or battery discharging and recharging patterns of real smartwatch users. Accordingly, we know little about the current practices for battery use and management of smartwatches. To address this, we conducted an online survey to examine usage behaviors of 59 smartwatch users and an in-depth analysis on the battery usage data from 17 Android Wear smartwatch users. We investigated the unique characteristics of smartwatches’ battery usage, users’ satisfaction and concerns, and recharging patterns through an online survey and data analysis on battery usage.
*This work was published in ISWC 2015 [ACM DL][Paper].
*We provide the collected data at [Web].

---

TalkBetter: Everyday Intervention Care for Children with Language Delay and Face-to-Face Interaction Monitoring Mobile Platform

Speech-language pathologists highlight that effective parent participation in everyday parent-child conversation is important to treat children's language delay. Our key inspiration behind the TalkBetter project was an emerging mobile platform for monitoring everyday face-to-face interaction, especially conversation monitoring, e.g., SocioPhone. SocioPhone monitors meta-linguistic contexts of conversation, such as turn-takings, prosodic features, a dominant participant, and pace using volume-topography-based method. We found that a meaningful number of guidelines for parents who have a child with language delay are related to meta-linguistic perspective, such as “Wait for the child talk back.”, and “Talk more slowly.” We designed and implemented TalkBetter, a mobile in-situ intervention service to help parents in daily parent-child conversation through real-time meta-linguistic analysis of ongoing conversations with speech-language pathologists.
*These works were published in MobiSys 2013 [ACM DL][Paper] and CSCW 2014 [ACM DL][Paper].
*The CSCW paper won the Best Paper Award.