Compact, worn computers with projected, on-skin touch interfaces have been a long-standing yet elusive goal, largely written off as science fiction. Such devices offer the potential to mitigate the significant human input/output bottleneck inherent in worn devices with small screens. In this work, we present the first, fully-functional and self-contained projection smartwatch implementation, containing the requisite compute, power, projection and touch-sensing capabilities.

Our watch offers roughly 40 cm² of interactive surface area – more than five times that of a typical smartwatch display. We demonstrate continuous 2D finger tracking with interactive, rectified graphics, transforming the arm into a touchscreen. We discuss our hardware and software implementation, as well as evaluation results regarding touch accuracy and projection visibility.


Xiao, R., Cao, T., Guo, N., Zhuo, J., Zhang, Y. and Harrison, C. 2018. LumiWatch: On-Arm Projected Graphics and Touch Input. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI ’18). ACM, New York, NY, USA, Paper 95, 11 pages. DOI: 10.1145/3173574.3173669





Previous articleHow plastic-eating bacteria actually work – a chemist explains
Next articleOpera now lets you beam content from your phone to your desktop
Robert Xiao
I’m a Ph.D. candidate in the Human-Computer Interaction Institute at Carnegie Mellon University, advised by Chris Harrison and Scott Hudson. I was born and raised in sunny Saskatoon. I completed a BMath in Computer Science and Combinatorics & Optimization at the University of Waterloo. I am a Qualcomm Innovation Fellow and Julie Payette NSERC Research Scholar. I routinely compete at DEFCON (1st place CTF 2016, 2017) and other security contests with my PPP teammates. I use my background in computer science and mathematics to craft novel interactive technologies driven by sensors, machine learning and computer vision. These technologies range from improving touchscreen input, to novel motion tracking systems, on-world projected interfaces, and much more.