Visualization as Intermediate Representations (VLAIR) for human activity recognition

Ai Jiang, Miguel Nacenta, Kasim Terzić, Juan Ye

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Ambient, binary, event-driven sensor data is useful for many human activity recognition applications such as smart homes and ambient-assisted living. These sensors are privacy-preserving, unobtrusive, inexpensive and easy to deploy in scenarios that require detection of simple activities such as going to sleep, and leaving the house. However, classification performance is still a challenge, especially when multiple people share the same space or when different activities take place in the same areas. To improve classification performance we develop what we call a Visualization as Intermediate Representations (VLAIR) approach. The main idea is to re-represent the data as visualizations (generated pixel images) in a similar way as how visualizations are created for humans to analyze and communicate data. Then we can feed these images to a convolutional neural network whose strength resides in extracting effective visual features. We have tested five variants (mappings) of the VLAIR approach and compared them to a collection of classifiers commonly used in classic human activity recognition. The best of the VLAIR approaches outperforms the best baseline, with strong advantage in recognising less frequent activities and distinguishing users and activities in common areas. We conclude the paper with a discussion on why and how VLAIR can be useful in human activity recognition scenarios and beyond.
Original languageEnglish
Title of host publicationPervasiveHealth '20:
Subtitle of host publicationProceedings of the 14th EAI International Conference on Pervasive Computing Technologies for Healthcare
EditorsSean A. Munson, Stephen M. Schueller
PublisherACM
Pages201-210
Number of pages10
ISBN (Print)9781450375320
DOIs
Publication statusPublished - 18 May 2020
Event14th EAI International Conference on Pervasive Computing Technologies for Healthcare (EAI PervasiveHealth 2020) - Atlanta, United States
Duration: 6 Oct 20208 Oct 2020
Conference number: 14

Conference

Conference14th EAI International Conference on Pervasive Computing Technologies for Healthcare (EAI PervasiveHealth 2020)
Abbreviated titleEAI PervasiveHealth 2020
Country/TerritoryUnited States
CityAtlanta
Period6/10/208/10/20

Keywords

  • Information visualization
  • Intermediate representations
  • Human activity recognition
  • Convolutional neural networks
  • Smart homes

Fingerprint

Dive into the research topics of 'Visualization as Intermediate Representations (VLAIR) for human activity recognition'. Together they form a unique fingerprint.

Cite this