VisuaLizations As Intermediate Representations (VLAIR): an approach for applying deep learning-based computer vision to non-image-based data

Ai Jiang*, Miguel A. Nacenta, Juan Ye

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Downloads (Pure)

Abstract

Deep learning algorithms increasingly support automated systems in areas such as human activity recognition and purchase recommendation. We identify a current trend in which data is transformed first into abstract visualizations and then processed by a computer vision deep learning pipeline. We call this VisuaLization As Intermediate Representation (VLAIR) and believe that it can be instrumental to support accurate recognition in a number of fields while also enhancing humans’ ability to interpret deep learning models for debugging purposes or in personal use. In this paper we describe the potential advantages of this approach and explore various visualization mappings and deep learning architectures. We evaluate several VLAIR alternatives for a specific problem (human activity recognition in an apartment) and show that VLAIR attains classification accuracy above classical machine learning algorithms and several other non-image-based deep learning algorithms with several data representations.
Original languageEnglish
Pages (from-to)35-50
Number of pages16
JournalVisual Informatics
Volume6
Issue number3
DOIs
Publication statusPublished - 24 Sept 2022

Keywords

  • Information visualization
  • Convolutional neural networks
  • Human activity recognition
  • Smart homes
  • Data representation
  • Intermediate representations
  • Interpretability
  • Machine learning
  • Deep learning

Fingerprint

Dive into the research topics of 'VisuaLizations As Intermediate Representations (VLAIR): an approach for applying deep learning-based computer vision to non-image-based data'. Together they form a unique fingerprint.

Cite this