Towards automatic cetacean photo-identification: a framework for fine-grain, few-shot learning in marine ecology

Cameron Trotter; Nick Wright; A. Stephen McGough; Matt Sharpe; Barbara Cheney; Monica Arso Civil; Reny Tyson Moore; Jason Allen; Per Berggren

doi:10.1109/bigdata55660.2022.10020942

Towards automatic cetacean photo-identification: a framework for fine-grain, few-shot learning in marine ecology

Cameron Trotter, Nick Wright, A. Stephen McGough, Matt Sharpe, Barbara Cheney, Monica Arso Civil, Reny Tyson Moore, Jason Allen, Per Berggren

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Photo-identification (photo-id) is one of the main non-invasive capture-recapture methods utilised by marine researchers for monitoring cetacean (dolphin, whale, and porpoise) populations. This method has historically been performed manually resulting in high workload and cost due to the vast number of images collected. Recently automated aids have been developed to help speed-up photo-id, although they are often disjoint in their processing and do not utilise all available identifying information. Work presented in this paper aims to create a fully automatic photo-id aid capable of providing most likely matches based on all available information without the need for data pre-processing such as cropping. This is achieved through a pipeline of computer vision models and post-processing techniques aimed at detecting cetaceans in unedited field imagery before passing them downstream for individual level catalogue matching. The system is capable of handling previously uncatalogued individuals and flagging these for investigation thanks to catalogue similarity comparison. We evaluate the system against multiple real-life photo-id catalogues, achieving mAP@IOU[0.5] = 0.91, 0.96 for the task of dorsal fin detection on catalogues from Tanzania and the UK respectively and 83.1, 97.5% top-10 accuracy for the task of individual classification on catalogues from the UK and USA.

Original language	English
Title of host publication	Proceedings
Subtitle of host publication	2022 IEEE International Conference on Big Data
Editors	Shusako Tsumoto, Yukio Ohsawa, Lei Chen, Dirk Van den Poel, Xiaohua Hu, Yoichi Motomura, Takuya Takagi, Lingfei Wu, Ying Xie, Akihiro Abe, Vijay Raghavan
Place of Publication	New York, NY
Publisher	IEEE
Pages	1942-1949
Number of pages	8
ISBN (Electronic)	9781665480468
DOIs	https://doi.org/10.1109/bigdata55660.2022.10020942
Publication status	Published - 17 Dec 2022
Event	2022 IEEE International Conference on Big Data (Big Data) - Duration: 17 Dec 2022 → 20 Dec 2022 https://ieeexplore.ieee.org/xpl/conhome/10020192/proceeding

Conference

Conference	2022 IEEE International Conference on Big Data (Big Data)
Period	17/12/22 → 20/12/22
Internet address	https://ieeexplore.ieee.org/xpl/conhome/10020192/proceeding

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/bigdata55660.2022.10020942

Cite this

Trotter, C., Wright, N., Stephen McGough, A., Sharpe, M., Cheney, B., Civil, M. A., Tyson Moore, R., Allen, J., & Berggren, P. (2022). Towards automatic cetacean photo-identification: a framework for fine-grain, few-shot learning in marine ecology. In S. Tsumoto, Y. Ohsawa, L. Chen, D. Van den Poel, X. Hu, Y. Motomura, T. Takagi, L. Wu, Y. Xie, A. Abe, & V. Raghavan (Eds.), Proceedings: 2022 IEEE International Conference on Big Data (pp. 1942-1949). IEEE. https://doi.org/10.1109/bigdata55660.2022.10020942

Trotter, Cameron ; Wright, Nick ; Stephen McGough, A. et al. / Towards automatic cetacean photo-identification : a framework for fine-grain, few-shot learning in marine ecology. Proceedings: 2022 IEEE International Conference on Big Data. editor / Shusako Tsumoto ; Yukio Ohsawa ; Lei Chen ; Dirk Van den Poel ; Xiaohua Hu ; Yoichi Motomura ; Takuya Takagi ; Lingfei Wu ; Ying Xie ; Akihiro Abe ; Vijay Raghavan. New York, NY : IEEE, 2022. pp. 1942-1949

@inproceedings{187611d200144c8b8f58cc5ec813be74,

title = "Towards automatic cetacean photo-identification: a framework for fine-grain, few-shot learning in marine ecology",

abstract = "Photo-identification (photo-id) is one of the main non-invasive capture-recapture methods utilised by marine researchers for monitoring cetacean (dolphin, whale, and porpoise) populations. This method has historically been performed manually resulting in high workload and cost due to the vast number of images collected. Recently automated aids have been developed to help speed-up photo-id, although they are often disjoint in their processing and do not utilise all available identifying information. Work presented in this paper aims to create a fully automatic photo-id aid capable of providing most likely matches based on all available information without the need for data pre-processing such as cropping. This is achieved through a pipeline of computer vision models and post-processing techniques aimed at detecting cetaceans in unedited field imagery before passing them downstream for individual level catalogue matching. The system is capable of handling previously uncatalogued individuals and flagging these for investigation thanks to catalogue similarity comparison. We evaluate the system against multiple real-life photo-id catalogues, achieving mAP@IOU[0.5] = 0.91, 0.96 for the task of dorsal fin detection on catalogues from Tanzania and the UK respectively and 83.1, 97.5% top-10 accuracy for the task of individual classification on catalogues from the UK and USA.",

author = "Cameron Trotter and Nick Wright and {Stephen McGough}, A. and Matt Sharpe and Barbara Cheney and Civil, {Monica Arso} and {Tyson Moore}, Reny and Jason Allen and Per Berggren",

year = "2022",

month = dec,

day = "17",

doi = "10.1109/bigdata55660.2022.10020942",

language = "English",

pages = "1942--1949",

editor = "Shusako Tsumoto and Yukio Ohsawa and Lei Chen and {Van den Poel}, Dirk and Xiaohua Hu and Yoichi Motomura and Takuya Takagi and Lingfei Wu and Ying Xie and Akihiro Abe and Vijay Raghavan",

booktitle = "Proceedings",

publisher = "IEEE",

note = "2022 IEEE International Conference on Big Data (Big Data) ; Conference date: 17-12-2022 Through 20-12-2022",

url = "https://ieeexplore.ieee.org/xpl/conhome/10020192/proceeding",

}

Trotter, C, Wright, N, Stephen McGough, A, Sharpe, M, Cheney, B, Civil, MA, Tyson Moore, R, Allen, J & Berggren, P 2022, Towards automatic cetacean photo-identification: a framework for fine-grain, few-shot learning in marine ecology. in S Tsumoto, Y Ohsawa, L Chen, D Van den Poel, X Hu, Y Motomura, T Takagi, L Wu, Y Xie, A Abe & V Raghavan (eds), Proceedings: 2022 IEEE International Conference on Big Data. IEEE, New York, NY, pp. 1942-1949, 2022 IEEE International Conference on Big Data (Big Data), 17/12/22. https://doi.org/10.1109/bigdata55660.2022.10020942

Towards automatic cetacean photo-identification: a framework for fine-grain, few-shot learning in marine ecology. / Trotter, Cameron; Wright, Nick; Stephen McGough, A. et al.
Proceedings: 2022 IEEE International Conference on Big Data. ed. / Shusako Tsumoto; Yukio Ohsawa; Lei Chen; Dirk Van den Poel; Xiaohua Hu; Yoichi Motomura; Takuya Takagi; Lingfei Wu; Ying Xie; Akihiro Abe; Vijay Raghavan. New York, NY: IEEE, 2022. p. 1942-1949.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Towards automatic cetacean photo-identification

T2 - 2022 IEEE International Conference on Big Data (Big Data)

AU - Trotter, Cameron

AU - Wright, Nick

AU - Stephen McGough, A.

AU - Sharpe, Matt

AU - Cheney, Barbara

AU - Civil, Monica Arso

AU - Tyson Moore, Reny

AU - Allen, Jason

AU - Berggren, Per

PY - 2022/12/17

Y1 - 2022/12/17

N2 - Photo-identification (photo-id) is one of the main non-invasive capture-recapture methods utilised by marine researchers for monitoring cetacean (dolphin, whale, and porpoise) populations. This method has historically been performed manually resulting in high workload and cost due to the vast number of images collected. Recently automated aids have been developed to help speed-up photo-id, although they are often disjoint in their processing and do not utilise all available identifying information. Work presented in this paper aims to create a fully automatic photo-id aid capable of providing most likely matches based on all available information without the need for data pre-processing such as cropping. This is achieved through a pipeline of computer vision models and post-processing techniques aimed at detecting cetaceans in unedited field imagery before passing them downstream for individual level catalogue matching. The system is capable of handling previously uncatalogued individuals and flagging these for investigation thanks to catalogue similarity comparison. We evaluate the system against multiple real-life photo-id catalogues, achieving mAP@IOU[0.5] = 0.91, 0.96 for the task of dorsal fin detection on catalogues from Tanzania and the UK respectively and 83.1, 97.5% top-10 accuracy for the task of individual classification on catalogues from the UK and USA.

AB - Photo-identification (photo-id) is one of the main non-invasive capture-recapture methods utilised by marine researchers for monitoring cetacean (dolphin, whale, and porpoise) populations. This method has historically been performed manually resulting in high workload and cost due to the vast number of images collected. Recently automated aids have been developed to help speed-up photo-id, although they are often disjoint in their processing and do not utilise all available identifying information. Work presented in this paper aims to create a fully automatic photo-id aid capable of providing most likely matches based on all available information without the need for data pre-processing such as cropping. This is achieved through a pipeline of computer vision models and post-processing techniques aimed at detecting cetaceans in unedited field imagery before passing them downstream for individual level catalogue matching. The system is capable of handling previously uncatalogued individuals and flagging these for investigation thanks to catalogue similarity comparison. We evaluate the system against multiple real-life photo-id catalogues, achieving mAP@IOU[0.5] = 0.91, 0.96 for the task of dorsal fin detection on catalogues from Tanzania and the UK respectively and 83.1, 97.5% top-10 accuracy for the task of individual classification on catalogues from the UK and USA.

UR - https://doi.org/10.1109/BigData55660.2022

U2 - 10.1109/bigdata55660.2022.10020942

DO - 10.1109/bigdata55660.2022.10020942

M3 - Conference contribution

SP - 1942

EP - 1949

BT - Proceedings

A2 - Tsumoto, Shusako

A2 - Ohsawa, Yukio

A2 - Chen, Lei

A2 - Van den Poel, Dirk

A2 - Hu, Xiaohua

A2 - Motomura, Yoichi

A2 - Takagi, Takuya

A2 - Wu, Lingfei

A2 - Xie, Ying

A2 - Abe, Akihiro

A2 - Raghavan, Vijay

PB - IEEE

CY - New York, NY

Y2 - 17 December 2022 through 20 December 2022

ER -

Trotter C, Wright N, Stephen McGough A, Sharpe M, Cheney B, Civil MA et al. Towards automatic cetacean photo-identification: a framework for fine-grain, few-shot learning in marine ecology. In Tsumoto S, Ohsawa Y, Chen L, Van den Poel D, Hu X, Motomura Y, Takagi T, Wu L, Xie Y, Abe A, Raghavan V, editors, Proceedings: 2022 IEEE International Conference on Big Data. New York, NY: IEEE. 2022. p. 1942-1949 doi: 10.1109/bigdata55660.2022.10020942

Towards automatic cetacean photo-identification: a framework for fine-grain, few-shot learning in marine ecology

Abstract

Conference

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this