DeepDOF: Deep learning extended depth-of-field microscope for fast and slide-free histology of surgical specimens

Abstract

Histopathology is the gold standard for cancer diagnosis, but histopathology slide preparation is expensive, time- and labor-intensive. Slide-free pathology with fluo- rescence microscopy could offer a faster and less costly alternative. However, rapidly imaging intact tissue with surface irregularities (∼ 200μm) is fundamentally con- strained by the intrinsic trade-off between resolution and depth-of-field (DOF). In this study, we present a novel computational microscope that can image intact spec- imens with cellular resolution without re-focusing. The system is designed to pro- vide real-time deep learning based histopathology of intact specimen using extended depth-of-field (DeepDOF). the DeepDOF microscope consists of a conventional microscope with the addition of a wavefront-encoding phase mask and a neural network that jointly extends the DOF while maintaining subcellular resolution. Leveraging advances in deep learning, we simultaneously designed and optimized the two key components in the DeepDOF network end-to-end. First, the optical layer simulates the phase mask that creates a depth-dependent and invertible point spread function (PSF). These PSF sections can encode surface texture/intensity information regardless of the surface topology. Sec- ond, an artificial intelligence-based digital layer is used to deconvolve and extract high resolution image information from the captured data. In this study, we trained the DeepDOF network and optimized the microscope design with a large image dataset consisting of varied imaging features from human histology to natural scenes. The optimized phase mask was then fabricated using reactive ion etching and inserted into the aperture plane of a 4x, 0.13 NA epi-fluorescence microscope, which was further integrated with an automated x-y sample stage for tissue mapping. We calibrated the depth dependent PSFs of the DeepDOF microscope using 1 μm fluorescent beads. By imaging resolution target, we show that the DeepDOF microscope can con- sistently resolve subcellular features within a 200 μm depth-of-field, thus allowing the visualization of nuclear morphology on highly irregular tissue surfaces without serial focusing. We validated DeepDOF microscope’s performance by imaging freshly resected and proflavine-stained porcine esophageal tissue and human oral tissue. Fur- thermore, we show that DeepDOF images reveal a variety of important diagnostic features confirmed by standard histopathology. In the long term, the DeepDOF mi- croscope can substantially contribute to histopathological assessment of intact biop- sies and surgical specimens, especially for intraoperative evaluation and in resource- constrained settings.