Paying per-label attention for multi-label extraction from radiology reports

Patrick Schrempf; Hannah Watson; Shadia Mikhael; Maciej Pajak; Matúš Falis; Aneta Lisowska; Keith W. Muir; David Harris-Birtill; Alison Q. O'Neil

doi:10.1007/978-3-030-61166-8_29

Paying per-label attention for multi-label extraction from radiology reports

Patrick Schrempf^*, Hannah Watson, Shadia Mikhael, Maciej Pajak, Matúš Falis, Aneta Lisowska, Keith W. Muir, David Harris-Birtill, Alison Q. O'Neil

^*Corresponding author for this work

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Training medical image analysis models requires large amounts of expertly annotated data which is time-consuming and expensive to obtain. Images are often accompanied by free-text radiology reports which are a rich source of information. In this paper, we tackle the automated extraction of structured labels from head CT reports for imaging of suspected stroke patients, using deep learning. Firstly, we propose a set of 31 labels which correspond to radiographic findings (e.g. hyperdensity) and clinical impressions (e.g. haemorrhage) related to neurological abnormalities. Secondly, inspired by previous work, we extend existing state-of-the-art neural network models with a label-dependent attention mechanism. Using this mechanism and simple synthetic data augmentation, we are able to robustly extract many labels with a single model, classified according to the radiologist's reporting (positive, uncertain, negative). This approach can be used in further research to effectively extract many labels from medical text.

Original language	English
Title of host publication	Interpretable and Annotation-Efficient Learning for Medical Image Computing
Subtitle of host publication	Third International Workshop, iMIMIC 2020, Second International Workshop, MIL3iD 2020, and 5th International Workshop, LABELS 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, Proceedings
Editors	Jaime Cardoso, Hien Van Nguyen, Nicholas Heller, Pedro Henriques Abreu, Ivana Isgum, Wilson Silva, Ricardo Cruz, Jose Pereira Amorim, Vishal Patel, Badri Roysam, Kevin Zhou, Steve Jiang, Ngan Le, Khoa Luu, Raphael Sznitman, Veronika Cheplygina, Diana Mateus, Emanuele Trucco, Samaneh Abbasi
Place of Publication	Cham
Publisher	Springer
Pages	277-289
Number of pages	13
ISBN (Electronic)	9783030611668
ISBN (Print)	9783030611651
DOIs	https://doi.org/10.1007/978-3-030-61166-8_29
Publication status	Published - 2020
Event	MICCAI Workshop on Large-scale Annotation of Biomedical data and Expert Label Synthesis 2020 - Lima (virtual), Peru Duration: 8 Oct 2020 → 8 Oct 2020 https://labels.tue-image.nl/

Publication series

Name	Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)
Publisher	Springer
Volume	12446 LNCS
ISSN (Print)	0302-9743

Workshop

Workshop	MICCAI Workshop on Large-scale Annotation of Biomedical data and Expert Label Synthesis 2020
Abbreviated title	LABELS 2020
Country/Territory	Peru
Period	8/10/20 → 8/10/20
Internet address	https://labels.tue-image.nl/

Keywords

NLP
Radiology report labelling
BERT

Access to Document

10.1007/978-3-030-61166-8_29

Schrempf-2020-Paying-per-label-attention-MICCAI-AAM
Copyright © 2020 Springer Nature Switzerland AG. This work has been made available online in accordance with publisher policies or with permission. Permission for further reuse of this content should be sought from the publisher or the rights holder. This is the author created accepted manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1007/978-3-030-61166-8_29.
Accepted author manuscript, 576 KB
Schrempf-2020-Paying-per-label-attention-MICCAI-SI-AAMAccepted author manuscript, 150 KB

ICAIRD: I-CAIRD: Industrial Centre for AI Research in Digital Diagnostics
Harris-Birtill, D. C. C. (PI) & Arandelovic, O. (CoI)
Technology Strategy Board
1/02/19 → 31/01/23
Project: Standard

David Cameron Christopher Harris-Birtill
- dcchbst-andrews.acuk
- School of Computer Science - Senior Lecturer
- Centre for Research into Ecological & Environmental Modelling
Person: Academic

Cite this

Schrempf, P., Watson, H., Mikhael, S., Pajak, M., Falis, M., Lisowska, A., Muir, K. W., Harris-Birtill, D., & O'Neil, A. Q. (2020). Paying per-label attention for multi-label extraction from radiology reports. In J. Cardoso, H. Van Nguyen, N. Heller, P. Henriques Abreu, I. Isgum, W. Silva, R. Cruz, J. Pereira Amorim, V. Patel, B. Roysam, K. Zhou, S. Jiang, N. Le, K. Luu, R. Sznitman, V. Cheplygina, D. Mateus, E. Trucco, & S. Abbasi (Eds.), Interpretable and Annotation-Efficient Learning for Medical Image Computing: Third International Workshop, iMIMIC 2020, Second International Workshop, MIL3iD 2020, and 5th International Workshop, LABELS 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, Proceedings (pp. 277-289). (Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics); Vol. 12446 LNCS). Springer. https://doi.org/10.1007/978-3-030-61166-8_29

Schrempf, Patrick ; Watson, Hannah ; Mikhael, Shadia et al. / Paying per-label attention for multi-label extraction from radiology reports. Interpretable and Annotation-Efficient Learning for Medical Image Computing: Third International Workshop, iMIMIC 2020, Second International Workshop, MIL3iD 2020, and 5th International Workshop, LABELS 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, Proceedings. editor / Jaime Cardoso ; Hien Van Nguyen ; Nicholas Heller ; Pedro Henriques Abreu ; Ivana Isgum ; Wilson Silva ; Ricardo Cruz ; Jose Pereira Amorim ; Vishal Patel ; Badri Roysam ; Kevin Zhou ; Steve Jiang ; Ngan Le ; Khoa Luu ; Raphael Sznitman ; Veronika Cheplygina ; Diana Mateus ; Emanuele Trucco ; Samaneh Abbasi. Cham : Springer, 2020. pp. 277-289 (Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)).

@inproceedings{b39b5af5ee084d4aaa407937a4e10b36,

title = "Paying per-label attention for multi-label extraction from radiology reports",

abstract = "Training medical image analysis models requires large amounts of expertly annotated data which is time-consuming and expensive to obtain. Images are often accompanied by free-text radiology reports which are a rich source of information. In this paper, we tackle the automated extraction of structured labels from head CT reports for imaging of suspected stroke patients, using deep learning. Firstly, we propose a set of 31 labels which correspond to radiographic findings (e.g. hyperdensity) and clinical impressions (e.g. haemorrhage) related to neurological abnormalities. Secondly, inspired by previous work, we extend existing state-of-the-art neural network models with a label-dependent attention mechanism. Using this mechanism and simple synthetic data augmentation, we are able to robustly extract many labels with a single model, classified according to the radiologist's reporting (positive, uncertain, negative). This approach can be used in further research to effectively extract many labels from medical text.",

keywords = "NLP, Radiology report labelling, BERT",

author = "Patrick Schrempf and Hannah Watson and Shadia Mikhael and Maciej Pajak and Mat{\'u}{\v s} Falis and Aneta Lisowska and Muir, {Keith W.} and David Harris-Birtill and O'Neil, {Alison Q.}",

note = "Funding: This work is part of the Industrial Centre for AI Research in digital Diagnostics (iCAIRD) which is funded by Innovate UK on behalf of UK Research and Innovation (UKRI) [project number: 104690]. ; MICCAI Workshop on Large-scale Annotation of Biomedical data and Expert Label Synthesis 2020, LABELS 2020 ; Conference date: 08-10-2020 Through 08-10-2020",

year = "2020",

doi = "10.1007/978-3-030-61166-8_29",

language = "English",

isbn = "9783030611651",

series = "Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)",

publisher = "Springer",

pages = "277--289",

editor = "Jaime Cardoso and {Van Nguyen}, Hien and Nicholas Heller and {Henriques Abreu}, Pedro and Ivana Isgum and Wilson Silva and Ricardo Cruz and {Pereira Amorim}, Jose and Vishal Patel and Badri Roysam and Kevin Zhou and Steve Jiang and Ngan Le and Khoa Luu and Raphael Sznitman and Veronika Cheplygina and Diana Mateus and Emanuele Trucco and Samaneh Abbasi",

booktitle = "Interpretable and Annotation-Efficient Learning for Medical Image Computing",

address = "Netherlands",

url = "https://labels.tue-image.nl/",

}

Schrempf, P, Watson, H, Mikhael, S, Pajak, M, Falis, M, Lisowska, A, Muir, KW, Harris-Birtill, D & O'Neil, AQ 2020, Paying per-label attention for multi-label extraction from radiology reports. in J Cardoso, H Van Nguyen, N Heller, P Henriques Abreu, I Isgum, W Silva, R Cruz, J Pereira Amorim, V Patel, B Roysam, K Zhou, S Jiang, N Le, K Luu, R Sznitman, V Cheplygina, D Mateus, E Trucco & S Abbasi (eds), Interpretable and Annotation-Efficient Learning for Medical Image Computing: Third International Workshop, iMIMIC 2020, Second International Workshop, MIL3iD 2020, and 5th International Workshop, LABELS 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, Proceedings. Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics), vol. 12446 LNCS, Springer, Cham, pp. 277-289, MICCAI Workshop on Large-scale Annotation of Biomedical data and Expert Label Synthesis 2020, Peru, 8/10/20. https://doi.org/10.1007/978-3-030-61166-8_29

Paying per-label attention for multi-label extraction from radiology reports. / Schrempf, Patrick; Watson, Hannah; Mikhael, Shadia et al.
Interpretable and Annotation-Efficient Learning for Medical Image Computing: Third International Workshop, iMIMIC 2020, Second International Workshop, MIL3iD 2020, and 5th International Workshop, LABELS 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, Proceedings. ed. / Jaime Cardoso; Hien Van Nguyen; Nicholas Heller; Pedro Henriques Abreu; Ivana Isgum; Wilson Silva; Ricardo Cruz; Jose Pereira Amorim; Vishal Patel; Badri Roysam; Kevin Zhou; Steve Jiang; Ngan Le; Khoa Luu; Raphael Sznitman; Veronika Cheplygina; Diana Mateus; Emanuele Trucco; Samaneh Abbasi. Cham: Springer, 2020. p. 277-289 (Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics); Vol. 12446 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Paying per-label attention for multi-label extraction from radiology reports

AU - Schrempf, Patrick

AU - Watson, Hannah

AU - Mikhael, Shadia

AU - Pajak, Maciej

AU - Falis, Matúš

AU - Lisowska, Aneta

AU - Muir, Keith W.

AU - Harris-Birtill, David

AU - O'Neil, Alison Q.

N1 - Funding: This work is part of the Industrial Centre for AI Research in digital Diagnostics (iCAIRD) which is funded by Innovate UK on behalf of UK Research and Innovation (UKRI) [project number: 104690].

PY - 2020

Y1 - 2020

N2 - Training medical image analysis models requires large amounts of expertly annotated data which is time-consuming and expensive to obtain. Images are often accompanied by free-text radiology reports which are a rich source of information. In this paper, we tackle the automated extraction of structured labels from head CT reports for imaging of suspected stroke patients, using deep learning. Firstly, we propose a set of 31 labels which correspond to radiographic findings (e.g. hyperdensity) and clinical impressions (e.g. haemorrhage) related to neurological abnormalities. Secondly, inspired by previous work, we extend existing state-of-the-art neural network models with a label-dependent attention mechanism. Using this mechanism and simple synthetic data augmentation, we are able to robustly extract many labels with a single model, classified according to the radiologist's reporting (positive, uncertain, negative). This approach can be used in further research to effectively extract many labels from medical text.

AB - Training medical image analysis models requires large amounts of expertly annotated data which is time-consuming and expensive to obtain. Images are often accompanied by free-text radiology reports which are a rich source of information. In this paper, we tackle the automated extraction of structured labels from head CT reports for imaging of suspected stroke patients, using deep learning. Firstly, we propose a set of 31 labels which correspond to radiographic findings (e.g. hyperdensity) and clinical impressions (e.g. haemorrhage) related to neurological abnormalities. Secondly, inspired by previous work, we extend existing state-of-the-art neural network models with a label-dependent attention mechanism. Using this mechanism and simple synthetic data augmentation, we are able to robustly extract many labels with a single model, classified according to the radiologist's reporting (positive, uncertain, negative). This approach can be used in further research to effectively extract many labels from medical text.

KW - NLP

KW - Radiology report labelling

KW - BERT

U2 - 10.1007/978-3-030-61166-8_29

DO - 10.1007/978-3-030-61166-8_29

M3 - Conference contribution

SN - 9783030611651

T3 - Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)

SP - 277

EP - 289

BT - Interpretable and Annotation-Efficient Learning for Medical Image Computing

A2 - Cardoso, Jaime

A2 - Van Nguyen, Hien

A2 - Heller, Nicholas

A2 - Henriques Abreu, Pedro

A2 - Isgum, Ivana

A2 - Silva, Wilson

A2 - Cruz, Ricardo

A2 - Pereira Amorim, Jose

A2 - Patel, Vishal

A2 - Roysam, Badri

A2 - Zhou, Kevin

A2 - Jiang, Steve

A2 - Le, Ngan

A2 - Luu, Khoa

A2 - Sznitman, Raphael

A2 - Cheplygina, Veronika

A2 - Mateus, Diana

A2 - Trucco, Emanuele

A2 - Abbasi, Samaneh

PB - Springer

CY - Cham

T2 - MICCAI Workshop on Large-scale Annotation of Biomedical data and Expert Label Synthesis 2020

Y2 - 8 October 2020 through 8 October 2020

ER -

Schrempf P, Watson H, Mikhael S, Pajak M, Falis M, Lisowska A et al. Paying per-label attention for multi-label extraction from radiology reports. In Cardoso J, Van Nguyen H, Heller N, Henriques Abreu P, Isgum I, Silva W, Cruz R, Pereira Amorim J, Patel V, Roysam B, Zhou K, Jiang S, Le N, Luu K, Sznitman R, Cheplygina V, Mateus D, Trucco E, Abbasi S, editors, Interpretable and Annotation-Efficient Learning for Medical Image Computing: Third International Workshop, iMIMIC 2020, Second International Workshop, MIL3iD 2020, and 5th International Workshop, LABELS 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, Proceedings. Cham: Springer. 2020. p. 277-289. (Lecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)). Epub 2020 Oct 2. doi: 10.1007/978-3-030-61166-8_29

Paying per-label attention for multi-label extraction from radiology reports

Abstract

Publication series

Workshop

Keywords

Access to Document

Fingerprint

Projects

ICAIRD: I-CAIRD: Industrial Centre for AI Research in Digital Diagnostics

Profiles

David Cameron Christopher Harris-Birtill

Cite this