SWAG demo: smart watch assisted gesture interaction for mixed reality head-mounted displays

Hyung-il Kim; Juyoung Lee; Hui Shyong Yeo; Aaron John Quigley; Woontack Woo

doi:10.1109/ISMAR-Adjunct.2018.00130

SWAG demo: smart watch assisted gesture interaction for mixed reality head-mounted displays

Hyung-il Kim, Juyoung Lee, Hui Shyong Yeo, Aaron John Quigley, Woontack Woo

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this demonstration, we will show a prototype system with sensor fusion approach to robustly track 6 degrees of freedom of hand movement and support intuitive hand gesture interaction and 3D object manipulation for Mixed Reality head-mounted displays. Robust tracking of hand and finger with egocentric camera remains a challenging problem, especially with self-occlusion – for example, when user tries to grab a virtual object in midair by closing the palm. Our approach leverages the use of a common smart watch worn on the wrist to provide a more reliable palm and wrist orientation data, while fusing the data with camera to achieve robust hand motion and orientation for interaction.

Original language	English
Title of host publication	Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	428-429
Number of pages	2
ISBN (Electronic)	9781538675922
DOIs	https://doi.org/10.1109/ISMAR-Adjunct.2018.00130
Publication status	Published - 25 Apr 2019
Event	17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018 - Munich, Germany Duration: 16 Oct 2018 → 20 Oct 2018 https://www.ismar2018.org/

Conference

Conference	17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018
Country/Territory	Germany
City	Munich
Period	16/10/18 → 20/10/18
Internet address	https://www.ismar2018.org/

Keywords

Augmented reality
Wearable computing
3D user interfaces
Hand interaction
Virtual 3D object manipulation

Access to Document

10.1109/ISMAR-Adjunct.2018.00130

Kim-2018-SWAG-demo-ISMAR-AAM
© 2018, IEEE. This work has been made available online in accordance with the publisher's policies. This is the author created accepted version manuscript following peer review and as such may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1109/ISMAR-Adjunct.2018.00130
Accepted author manuscript, 477 KB

Cite this

Kim, H., Lee, J., Yeo, H. S., Quigley, A. J., & Woo, W. (2019). SWAG demo: smart watch assisted gesture interaction for mixed reality head-mounted displays. In Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018 (pp. 428-429). Article 8699201 Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISMAR-Adjunct.2018.00130

@inproceedings{e6662490bb204dec8e2e25ca8b30b991,

title = "SWAG demo: smart watch assisted gesture interaction for mixed reality head-mounted displays",

abstract = "In this demonstration, we will show a prototype system with sensor fusion approach to robustly track 6 degrees of freedom of hand movement and support intuitive hand gesture interaction and 3D object manipulation for Mixed Reality head-mounted displays. Robust tracking of hand and finger with egocentric camera remains a challenging problem, especially with self-occlusion – for example, when user tries to grab a virtual object in midair by closing the palm. Our approach leverages the use of a common smart watch worn on the wrist to provide a more reliable palm and wrist orientation data, while fusing the data with camera to achieve robust hand motion and orientation for interaction.",

keywords = "Augmented reality, Wearable computing, 3D user interfaces, Hand interaction, Virtual 3D object manipulation",

author = "Hyung-il Kim and Juyoung Lee and Yeo, \{Hui Shyong\} and Quigley, \{Aaron John\} and Woontack Woo",

year = "2019",

month = apr,

day = "25",

doi = "10.1109/ISMAR-Adjunct.2018.00130",

language = "English",

pages = "428--429",

booktitle = "Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018 ; Conference date: 16-10-2018 Through 20-10-2018",

url = "https://www.ismar2018.org/",

}

Kim, H, Lee, J, Yeo, HS, Quigley, AJ & Woo, W 2019, SWAG demo: smart watch assisted gesture interaction for mixed reality head-mounted displays. in Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018., 8699201, Institute of Electrical and Electronics Engineers Inc., pp. 428-429, 17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018, Munich, Germany, 16/10/18. https://doi.org/10.1109/ISMAR-Adjunct.2018.00130

SWAG demo: smart watch assisted gesture interaction for mixed reality head-mounted displays. / Kim, Hyung-il; Lee, Juyoung; Yeo, Hui Shyong et al.
Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018. Institute of Electrical and Electronics Engineers Inc., 2019. p. 428-429 8699201.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - SWAG demo

T2 - 17th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018

AU - Kim, Hyung-il

AU - Lee, Juyoung

AU - Yeo, Hui Shyong

AU - Quigley, Aaron John

AU - Woo, Woontack

PY - 2019/4/25

Y1 - 2019/4/25

N2 - In this demonstration, we will show a prototype system with sensor fusion approach to robustly track 6 degrees of freedom of hand movement and support intuitive hand gesture interaction and 3D object manipulation for Mixed Reality head-mounted displays. Robust tracking of hand and finger with egocentric camera remains a challenging problem, especially with self-occlusion – for example, when user tries to grab a virtual object in midair by closing the palm. Our approach leverages the use of a common smart watch worn on the wrist to provide a more reliable palm and wrist orientation data, while fusing the data with camera to achieve robust hand motion and orientation for interaction.

AB - In this demonstration, we will show a prototype system with sensor fusion approach to robustly track 6 degrees of freedom of hand movement and support intuitive hand gesture interaction and 3D object manipulation for Mixed Reality head-mounted displays. Robust tracking of hand and finger with egocentric camera remains a challenging problem, especially with self-occlusion – for example, when user tries to grab a virtual object in midair by closing the palm. Our approach leverages the use of a common smart watch worn on the wrist to provide a more reliable palm and wrist orientation data, while fusing the data with camera to achieve robust hand motion and orientation for interaction.

KW - Augmented reality

KW - Wearable computing

KW - 3D user interfaces

KW - Hand interaction

KW - Virtual 3D object manipulation

UR - https://www.scopus.com/pages/publications/85065528667

U2 - 10.1109/ISMAR-Adjunct.2018.00130

DO - 10.1109/ISMAR-Adjunct.2018.00130

M3 - Conference contribution

AN - SCOPUS:85065528667

SP - 428

EP - 429

BT - Adjunct Proceedings - 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 16 October 2018 through 20 October 2018

ER -

SWAG demo: smart watch assisted gesture interaction for mixed reality head-mounted displays

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this