Impact of different mammography systems on artificial intelligence performance in breast cancer screening

Clarisse F. de Vries*, Samantha J. Colosimo, Roger T. Staff, Jaroslaw A. Dymiter, Joseph Yearsley, Deirdre Dinneen, Moragh Boyle, David J. Harrison, Lesley A. Anderson, Gerald Lip, Corri Black, Alison D. Murray, Katie Wilde, James D. Blackwood, Claire Butterly, John Zurowski, Jon Eilbeck, Colin McSkimming

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

15 Citations (Scopus)
9 Downloads (Pure)

Abstract

Artificial intelligence (AI) tools may assist breast screening mammography programs, but limited evidence supports their generalizability to new settings. This retrospective study used a 3-year dataset (April 1, 2016-March 31, 2019) from a U.K. regional screening program. The performance of a commercially available breast screening AI algorithm was assessed with a prespecified and site-specific decision threshold to evaluate whether its performance was transferable to a new clinical site. The dataset consisted of women (aged approximately 50-70 years) who attended routine screening, excluding self-referrals, those with complex physical requirements, those who had undergone a previous mastectomy, and those who underwent screening that had technical recalls or did not have the four standard image views. In total, 55916 screening attendees (mean age, 60 years ± 6 [SD]) met the inclusion criteria. The prespecified threshold resulted in high recall rates (48.3%, 21929 of 45444), which reduced to 13.0% (5896 of 45444) following threshold calibration, closer to the observed service level (5.0%, 2774 of 55916). Recall rates also increased approximately threefold following a software upgrade on the mammography equipment, requiring per-software version thresholds. Using software-specific thresholds, the AI algorithm would have recalled 277 of 303 (91.4%) screen-detected cancers and 47 of 138 (34.1%) interval cancers. AI performance and thresholds should be validated for new clinical settings before deployment, while quality assurance systems should monitor AI performance for consistency.
Original languageEnglish
Article numbere220146
Number of pages8
JournalRadiology: Artificial Intelligence
Volume5
Issue number3
Early online date22 Mar 2023
DOIs
Publication statusPublished - 1 May 2023

Keywords

  • Breast
  • Screening
  • Mammography
  • Computer applications: detection/diagnosis
  • Neoplasms: primary
  • Technology assessment

Fingerprint

Dive into the research topics of 'Impact of different mammography systems on artificial intelligence performance in breast cancer screening'. Together they form a unique fingerprint.

Cite this