Estimating full-field displacement in biological images using deep learning

The estimation of full-field displacement between biological images or in videos is important for quantitative analyses of motion, dynamics and biophysics. However, the often-weak signals, poor contrast and a multitude of noise processes typical to microscopy make this a formidable challenge for many contemporary methods. Here, we present a deep-learning method, termed Displacement Estimation FOR Microscopy (DEFORM-Net), that outperforms traditional digital image correlation and optical flow methods, as well as recent learned approaches, offering simultaneous high accuracy, spatial sampling and speed. DEFORM-Net is experimentally unsupervised, relying on displacement simulation based on a random fractal Perlin-noise process and optimised training loss functions, without experimental ground truths. We demonstrate DEFORM-Net on real biological videos of beating neonatal mouse cardiomyocytes and pulsed contractions in Drosophila pupae, and in various microscopy modalities. We provide DEFORM-Net as open source, including inference in the ImageJ/FIJI platform, to empower new quantitative applications in biology and medicine.