ROMA: run-time object detection to maximize real-time accuracy

JunKyu Lee; Blesson Varghese; Hans Vandierendonck

doi:10.1109/wacv56688.2023.00634

ROMA: run-time object detection to maximize real-time accuracy

JunKyu Lee, Blesson Varghese, Hans Vandierendonck

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

16 Downloads (Pure)

Abstract

This paper analyzes the effects of dynamically varying video contents and detection latency on the real-time detection accuracy of a detector and proposes a new run-time accuracy variation model, ROMA, based on the findings from the analysis. ROMA is designed to select an optimal detector out of a set of detectors in real time without label information to maximize real-time object detection accuracy. ROMA utilizing four YOLOv4 detectors on an NVIDIA Jetson Nano shows real-time accuracy improvements by 4 to 37% for a scenario of dynamically varying video con-tents and detection latency consisting of MOT17Det and MOT20Det datasets, compared to individual YOLOv4 detectors and two state-of-the-art runtime techniques.

Original language	English
Title of host publication	Proceedings 2023 IEEE/CVF winter conference on applications of computer vision (WACV)
Subtitle of host publication	3-7 January 2023, Waikoloa, Hawaii
Editors	Tamara Berg, Ryan Farrell
Place of Publication	Piscataway, NJ
Publisher	IEEE
Pages	6394-6403
Number of pages	10
ISBN (Electronic)	9781665493468
ISBN (Print)	9781665493475
DOIs	https://doi.org/10.1109/wacv56688.2023.00634
Publication status	Published - 6 Feb 2023
Event	IEEE/CVF Winter Conference on Applications of Computer Vision - Waikoloa, United States Duration: 3 Jan 2023 → 7 Jan 2023 https://wacv2023.thecvf.com https://doi.org/10.1109/WACV56688.2023

Publication series

Name	2023 IEEE/CVF winter conference on applications of computer vision (WACV)
ISSN (Print)	2472-6737
ISSN (Electronic)	2642-9381

Conference

Conference	IEEE/CVF Winter Conference on Applications of Computer Vision
Abbreviated title	WACV 2023
Country/Territory	United States
City	Waikoloa
Period	3/01/23 → 7/01/23
Internet address	https://wacv2023.thecvf.com https://doi.org/10.1109/WACV56688.2023

Keywords

Applications: embedded sensing/real-time techniques

Access to Document

10.1109/wacv56688.2023.00634

Lee_2023_IEEE-CVF_ROMA_AAM
Copyright © 2023 IEEE. This work has been made available online in accordance with publisher policies or with permission. Permission for further reuse of this content should be sought from the publisher or the rights holder. This is the author created accepted manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1109/wacv56688.2023.00634.
Accepted author manuscript, 993 KB

https://openaccess.thecvf.com/content/WACV2023/papers/Lee_ROMA_Run-Time_Object_Detection_To_Maximize_Real-Time_Accuracy_WACV_2023_paper.pdf

Cite this

Lee, J., Varghese, B., & Vandierendonck, H. (2023). ROMA: run-time object detection to maximize real-time accuracy. In T. Berg, & R. Farrell (Eds.), Proceedings 2023 IEEE/CVF winter conference on applications of computer vision (WACV): 3-7 January 2023, Waikoloa, Hawaii (pp. 6394-6403). Article 10030225 (2023 IEEE/CVF winter conference on applications of computer vision (WACV)). IEEE. https://doi.org/10.1109/wacv56688.2023.00634

Lee, JunKyu ; Varghese, Blesson ; Vandierendonck, Hans. / ROMA : run-time object detection to maximize real-time accuracy. Proceedings 2023 IEEE/CVF winter conference on applications of computer vision (WACV): 3-7 January 2023, Waikoloa, Hawaii. editor / Tamara Berg ; Ryan Farrell. Piscataway, NJ : IEEE, 2023. pp. 6394-6403 (2023 IEEE/CVF winter conference on applications of computer vision (WACV)).

@inproceedings{a7e4e0ac4eb74855bfd229de84b0393b,

title = "ROMA: run-time object detection to maximize real-time accuracy",

abstract = "This paper analyzes the effects of dynamically varying video contents and detection latency on the real-time detection accuracy of a detector and proposes a new run-time accuracy variation model, ROMA, based on the findings from the analysis. ROMA is designed to select an optimal detector out of a set of detectors in real time without label information to maximize real-time object detection accuracy. ROMA utilizing four YOLOv4 detectors on an NVIDIA Jetson Nano shows real-time accuracy improvements by 4 to 37% for a scenario of dynamically varying video con-tents and detection latency consisting of MOT17Det and MOT20Det datasets, compared to individual YOLOv4 detectors and two state-of-the-art runtime techniques.",

keywords = "Applications: embedded sensing/real-time techniques",

author = "JunKyu Lee and Blesson Varghese and Hans Vandierendonck",

note = "Funding: This work is supported by the Engineering and Physical Sciences Research Council under grant agreement EP/T022345/1 and by CHIST-ERA under grant agreement CHIST-ERA-18-SDCDN-002 (DiPET).; IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2023 ; Conference date: 03-01-2023 Through 07-01-2023",

year = "2023",

month = feb,

day = "6",

doi = "10.1109/wacv56688.2023.00634",

language = "English",

isbn = "9781665493475",

series = "2023 IEEE/CVF winter conference on applications of computer vision (WACV)",

publisher = "IEEE",

pages = "6394--6403",

editor = "Tamara Berg and Ryan Farrell",

booktitle = "Proceedings 2023 IEEE/CVF winter conference on applications of computer vision (WACV)",

url = "https://wacv2023.thecvf.com, https://doi.org/10.1109/WACV56688.2023",

}

Lee, J, Varghese, B & Vandierendonck, H 2023, ROMA: run-time object detection to maximize real-time accuracy. in T Berg & R Farrell (eds), Proceedings 2023 IEEE/CVF winter conference on applications of computer vision (WACV): 3-7 January 2023, Waikoloa, Hawaii., 10030225, 2023 IEEE/CVF winter conference on applications of computer vision (WACV), IEEE, Piscataway, NJ, pp. 6394-6403, IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, Hawaii, United States, 3/01/23. https://doi.org/10.1109/wacv56688.2023.00634

ROMA: run-time object detection to maximize real-time accuracy. / Lee, JunKyu; Varghese, Blesson; Vandierendonck, Hans.
Proceedings 2023 IEEE/CVF winter conference on applications of computer vision (WACV): 3-7 January 2023, Waikoloa, Hawaii. ed. / Tamara Berg; Ryan Farrell. Piscataway, NJ: IEEE, 2023. p. 6394-6403 10030225 (2023 IEEE/CVF winter conference on applications of computer vision (WACV)).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - ROMA

T2 - IEEE/CVF Winter Conference on Applications of Computer Vision

AU - Lee, JunKyu

AU - Varghese, Blesson

AU - Vandierendonck, Hans

N1 - Funding: This work is supported by the Engineering and Physical Sciences Research Council under grant agreement EP/T022345/1 and by CHIST-ERA under grant agreement CHIST-ERA-18-SDCDN-002 (DiPET).

PY - 2023/2/6

Y1 - 2023/2/6

N2 - This paper analyzes the effects of dynamically varying video contents and detection latency on the real-time detection accuracy of a detector and proposes a new run-time accuracy variation model, ROMA, based on the findings from the analysis. ROMA is designed to select an optimal detector out of a set of detectors in real time without label information to maximize real-time object detection accuracy. ROMA utilizing four YOLOv4 detectors on an NVIDIA Jetson Nano shows real-time accuracy improvements by 4 to 37% for a scenario of dynamically varying video con-tents and detection latency consisting of MOT17Det and MOT20Det datasets, compared to individual YOLOv4 detectors and two state-of-the-art runtime techniques.

AB - This paper analyzes the effects of dynamically varying video contents and detection latency on the real-time detection accuracy of a detector and proposes a new run-time accuracy variation model, ROMA, based on the findings from the analysis. ROMA is designed to select an optimal detector out of a set of detectors in real time without label information to maximize real-time object detection accuracy. ROMA utilizing four YOLOv4 detectors on an NVIDIA Jetson Nano shows real-time accuracy improvements by 4 to 37% for a scenario of dynamically varying video con-tents and detection latency consisting of MOT17Det and MOT20Det datasets, compared to individual YOLOv4 detectors and two state-of-the-art runtime techniques.

KW - Applications: embedded sensing/real-time techniques

UR - https://doi.org/10.1109/WACV56688.2023

UR - https://doi.org/10.48550/arXiv.2210.16083

U2 - 10.1109/wacv56688.2023.00634

DO - 10.1109/wacv56688.2023.00634

M3 - Conference contribution

SN - 9781665493475

T3 - 2023 IEEE/CVF winter conference on applications of computer vision (WACV)

SP - 6394

EP - 6403

BT - Proceedings 2023 IEEE/CVF winter conference on applications of computer vision (WACV)

A2 - Berg, Tamara

A2 - Farrell, Ryan

PB - IEEE

CY - Piscataway, NJ

Y2 - 3 January 2023 through 7 January 2023

ER -

Lee J, Varghese B, Vandierendonck H. ROMA: run-time object detection to maximize real-time accuracy. In Berg T, Farrell R, editors, Proceedings 2023 IEEE/CVF winter conference on applications of computer vision (WACV): 3-7 January 2023, Waikoloa, Hawaii. Piscataway, NJ: IEEE. 2023. p. 6394-6403. 10030225. (2023 IEEE/CVF winter conference on applications of computer vision (WACV)). Epub 2023 Feb 6. doi: 10.1109/wacv56688.2023.00634

ROMA: run-time object detection to maximize real-time accuracy

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this