SpeCam: sensing surface color and material with the front-facing camera of mobile device

Hui Shyong Yeo, Juyoung Lee, Andrea Bianchi, David Harris-Birtill, Aaron John Quigley

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Citations (Scopus)
1 Downloads (Pure)

Abstract

SpeCam is a lightweight surface color and material sensing approach for mobile devices which only uses the front-facing camera and the display as a multi-spectral light source. We leverage the natural use of mobile devices (placing it face-down) to detect the material underneath and therefore infer the location or placement of the device. SpeCam can then be used to support discreet micro-interactions to avoid the numerous distractions that users daily face with today's mobile devices. Our two-parts study shows that SpeCam can i) recognize colors in the HSB space with 10 degrees apart near the 3 dominant colors and 4 degrees otherwise and ii) 30 types of surface materials with 99% accuracy. These findings are further supported by a spectroscopy study. Finally, we suggest a series of applications based on simple mobile micro-interactions suitable for using the phone when placed face-down.
Original languageEnglish
Title of host publicationProceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17)
Place of PublicationNew York, NY
PublisherACM
Pages1-9
Number of pages9
ISBN (Print)9781450350754
DOIs
Publication statusPublished - 4 Sept 2017
Event19th International Conference on Human-Computer Interaction with Mobile Devices and Services - Aula der Wissenschaft – Hall of Science, Vienna, Austria
Duration: 4 Sept 20177 Sept 2017
Conference number: 19
https://mobilehci.acm.org/2017/index.html

Conference

Conference19th International Conference on Human-Computer Interaction with Mobile Devices and Services
Abbreviated titleMobileHCI
Country/TerritoryAustria
CityVienna
Period4/09/177/09/17
Internet address

Keywords

  • Surface detection
  • Material detection
  • Discreet interaction

Fingerprint

Dive into the research topics of 'SpeCam: sensing surface color and material with the front-facing camera of mobile device'. Together they form a unique fingerprint.

Cite this