The colour of time: detecting glioma IDH mutation status in MRI through pseudo-coloured transfer learning

Background: Glioma is the most common brain cancer and is conventionally diagnosed with MR imaging. Its prognosis and treatment depend on the tumour genetic subtype. However, tumour genotyping is invasive, requiring a sample of tumour tissue; a noninvasive method to determine glioma subtype from an image would be a valuable addition to the oncology toolbox. Necessary restrictions on access to clinical data make developing medical applications challenging. Radiogenomics is especially challenging, since it requires paired imaging and genotype data.
Aims: We investigate whether classification models, pre-trained on natural scene images before being finetuned on MR images to determine glioma subtype, can outperform models trained from scratch on larger private medical datasets. We investigate the most effective way of applying the MR sequences to the colour model.
Methods: The T1, contrast enhanced T1, T2 and FLAIR sequences (defined by their different repetition, echo and inversion times) are used as inputs to the colour channels, allowing the use of preexisting natural scene models. A hyperparameter search determined the optimum parameters. Two pretrained CNN models (VGG16 and ResNext) were finetuned and compared across 24 pseudo-colour permutations and 4 grey monocolour configurations to explore effects on performance from combinations of MR sequence and colour channel.
Results: Our best model exceeds the baseline from literature, achieving 88.1% accuracy, 0.935 AUC and 0.819 F1 score on a held out test set.
Conclusions: Classification of genetic markers in volumetric images can be undertaken effectively and efficiently with models pretrained on 2D natural scene images finetuned for the imaging genomics task. Crafting a custom 3D volumetric model from scratch is not always necessary.