Paying per-label attention for multi-label extraction from radiology reports

Patrick Schrempf*, Hannah Watson, Shadia Mikhael, Maciej Pajak, Matúš Falis, Aneta Lisowska, Keith W. Muir, David Harris-Birtill, Alison Q. O'Neil

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)
5 Downloads (Pure)


Training medical image analysis models requires large amounts of expertly annotated data which is time-consuming and expensive to obtain. Images are often accompanied by free-text radiology reports which are a rich source of information. In this paper, we tackle the automated extraction of structured labels from head CT reports for imaging of suspected stroke patients, using deep learning. Firstly, we propose a set of 31 labels which correspond to radiographic findings (e.g. hyperdensity) and clinical impressions (e.g. haemorrhage) related to neurological abnormalities. Secondly, inspired by previous work, we extend existing state-of-the-art neural network models with a label-dependent attention mechanism. Using this mechanism and simple synthetic data augmentation, we are able to robustly extract many labels with a single model, classified according to the radiologist's reporting (positive, uncertain, negative). This approach can be used in further research to effectively extract many labels from medical text.
Original languageEnglish
Title of host publicationInterpretable and Annotation-Efficient Learning for Medical Image Computing
Subtitle of host publicationThird International Workshop, iMIMIC 2020, Second International Workshop, MIL3iD 2020, and 5th International Workshop, LABELS 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, Proceedings
EditorsJaime Cardoso, Hien Van Nguyen, Nicholas Heller, Pedro Henriques Abreu, Ivana Isgum, Wilson Silva, Ricardo Cruz, Jose Pereira Amorim, Vishal Patel, Badri Roysam, Kevin Zhou, Steve Jiang, Ngan Le, Khoa Luu, Raphael Sznitman, Veronika Cheplygina, Diana Mateus, Emanuele Trucco, Samaneh Abbasi
Place of PublicationCham
Number of pages13
ISBN (Electronic)9783030611668
ISBN (Print)9783030611651
Publication statusPublished - 2020
EventMICCAI Workshop on Large-scale Annotation of Biomedical data and Expert Label Synthesis 2020 - Lima (virtual), Peru
Duration: 8 Oct 20208 Oct 2020

Publication series

NameLecture Notes in Computer Science (including subseries Image Processing, Computer Vision, Pattern Recognition, and Graphics)
Volume12446 LNCS
ISSN (Print)0302-9743


WorkshopMICCAI Workshop on Large-scale Annotation of Biomedical data and Expert Label Synthesis 2020
Abbreviated titleLABELS 2020
Internet address


  • NLP
  • Radiology report labelling
  • BERT


Dive into the research topics of 'Paying per-label attention for multi-label extraction from radiology reports'. Together they form a unique fingerprint.

Cite this