Table of Contents

Deep Learning-Based Neuron Detection in Brain CLARITY Imaging

— Prerna Singh 2020/05/09 14:06

Summary

Our objective is to develop a 3-dimensional convolutional neural network (CNN) that can predict, with increased precision and accuracy when compared to other models, how many fluorescent neurons are present within a section of a brain imaged with CLARITY.

We propose a six-layer 3-dimensional convolutional neural network (CNN) to count the number of fluorescent neurons present within a section of a brain imaged with CLARITY, a critical tool for the connectome project5, enabling the exploration of important neuroscience questions. Our results, cross-validated over different splits show accuracy scores consistently greater than 95% with area under the ROC curves consistently greater than .9. Given the capability to predict with greater than 95% accuracy, we believe that deep convolutional neural networks are capable of detecting neurons in CLARITY images. Potentially, a similar model could also be used for other biomarker detection tasks.

Background, Specific Aims, and Significance

Clear lipid-exchanged acrylamide-hybridized rigid in-situ-compatible tissue hydrogel (CLARITY) imaging is unique in that it makes brain tissue transparent, allowing for a complete 3-dimensional representation of the brain in a 2-dimensional scan. More generally, specific antibodies can be engineered to bind to cells and cause fluorescence, based on which cells scientists are studying within the brain. Being able to look at the brain in three dimensions allows scientists to focus on certain structures without losing focus of the whole brain in the background.2

CLARITY is a critical tool for the connectome project5, which looks at how neural cells fire together, within a hierarchy of neural activation patterns in the brain. The overall goal of the connectome project is to develop a comprehensive map of neural connections in the brain. CLARITY can give a lot of insight into that by seeing how different parts of the brain activate and fluoresce with neurons in other locations in the brain, which has not been possible given the intrinsic limitations of previous kinds of imaging, which give only a 2-dimensional picture of cells in the brain. The experiments try to understand how different cell types in the brain link their activity and pathways. Through CLARITY scans, cellular logic and behavior can be understood through cell-type specific excitation throughout the brain.1

In any of these different applications of CLARITY, processing each scan usually entails annotating and counting cells by hand. These images can be fairly large, and for instance, in the data set used in this project, there are ~1000 slices per brain. In practice, this can be very time consuming and inefficient, given the amount of information that can be extracted by these scans. Previous methods attempted to solve this problem include: filtering, template matching, and blob detection, which had a maximal accuracy of ~59%. These are all non-learning methods, which asks the question if a deep learning approach would have a better performance. Development of a 3-D CNN to count the fluorescing cells, will streamline the processing of CLARITY scans, allowing for critical information from these scans to be available almost immediately.

Deliverables

Technical Approach

The project workflow is split into the following main components:

  1. Design and Implementation of 3-dimensional CNN Model
  2. First training and validation of 3-dimensional CNN training on reduced dataset (1 brain)
  3. Large scale training on complete dataset (8 brains) as well as CNN validation

Design and Implementation of 3-dimensional CNN Model

The 3D CNN Layers proposed are based on the paper “Automatic Detection of Cerebral Microbleeds from MR Images via 3D Convolutional Neural Networks” that detects cerebral microbleeds in 3D brain scans and outperformed previous attempts at this problem with increased accuracy and sensitivity and fewer false positives.3 The CNN has three convolutional layers, one maxpool layer, and ends with two fully connected layers. The advantage of using a 3D CNN is that it can take advantage of the spatial contextual information to extract more representative high-level features for detecting these neurons.3

The proposed 3D CNN can take a 3D input of any size greater than 12x12x12 pixels and reports a score ranging from 0 to 1 whether it is a fluorescing neuron (1) or not (0). From here, a sensitivity threshold will be determined based on optimization of performance metrics (precision, accuracy, sensitivity, recall, etc.) that will lead to a binary decision of fluorescing neuron (1) or not (0).

First training and validation of 3-dimensional CNN training on reduced dataset

The complete dataset for this project are CLARITY images for eight full brains. It contains ~1500 two-dimensional images, with 2560×2160 pixels each, taken at different depths of the brain. These eight brains contain annotations for the coordinates of some of the fluorescing cells, totaling approximately 2000 annotated cells. Note that not all the fluorescing cells are annotated.

The figure above portrays the workflow of training the CNN. Based on my empirical measurements and previous work, it was determined that a voxel size of 12x12x12 pixels will contain the entire cell. Thus, 12x12x12 voxels are used for training, with both positive and negative controls. Voxels that contain a fluorescing cell were used as positive training data for the CNN. To generate negative samples (non-fluorescing cell) to train the CNN, ten times the amount of non-cell regions were generated by randomly sampling 12x12x12 regions that are within 6-20 pixels of regions centered on the cell annotations. This was done because the number of fluorescing cells within the brain are actually a very small fraction of all cells.

Due to the variation in brain intensity in the scans, before extracting any data for training, a brain illumination correction filter, N4ITK6 was applied to the data. Without the N4ITK filter, it was quite hard to visualize the fluorescent cells, even with the human eye, since there was so much variance in the background. By applying a brain illumination correction filter to the data, we can see that it becomes much easier to distinguish the fluorescent cells. The figure below shows an example of using the illumination correction filter.

Large scale training on complete dataset (8 brains) as well as CNN validation

We used all eight brains for training the final model. However, in order for the validation to be unbiased, two different validations were performed, by excluding some training data and using it for testing. The first is coarse validation, where metrics such as area under the ROC (AUC) and PR (APR) curves, precision, recall, and accuracy will be reported over random splits of training vs. testing data. Each of the splits were also retrained with some false positives and all of the false negative results. The second validation is full cross validation, where the same metrics will be reported and averaged over four folds. In each fold, six brains were used for training and the remaining two for testing.

Coarse Validation

Cross Validation

False Positives from Split 2 Retrained

Discussion of Results and Significance

Given the results we can conclude that the proposed model actually performs even better than the statistics shown. The false positives from Split 2 retrained are displayed above. 14 of these 19 voxels do actually seem to contain a cell. This can be attributed to the fact that the training data had only a fraction of annotated cell regions and there were many more cell regions that were not annotated. In the way the non-cell training data was generated, some of the non-cell regions actually capture cells, and the model does a good job of classifying those as such, based on the adjusted FP column, which accounts for the incorrectly labeled training data. Given this, we can assume that all of the metrics used to quantify the quality of the 3D CNN are actually better than what is shown in the tables, ROC, and PR curves.

The proposed 3D CNN model surpasses the accuracy of the previous model significantly. The results of the coarse and cross-validation show accuracy scores consistently greater than 95% compared to 59% by previous methods7. This can lead to major breakthroughs in neuroscience research. To be able to analyze any brain scan, identifying neurons is critical. Given the capability to predict with greater than 95% accuracy, we believe that deep convolutional neural networks are capable of detecting neurons in this kind of imaging technique. Potentially, a similar model could also be used for other biomarker detection tasks.

Dependencies

Computational Infrastructure

Due to the immense computational requirements necessary to train a 3-dimensional convolutional neural network, MARCC (Maryland Advanced Research Computing Center https://www.marcc.jhu.edu/) resources were utilized. Blue Crab is the main cluster at MARCC with over 23,000 cores (June 2018) and a combined theoretical performance of over 1.4 PFLOPs. 72 GPU nodes were used to complete the training and validation of the proposed model. The GPU nodes are Dell PowerEdge R730 servers with dual Intel Haswell Xeon E5-2680v3 (12 core, 2.5 GHz, 120W), 128 GB of 2133 MHz DDR4 RAM. (AVX frequency: 2.1GHz) and two Nvidia K80s per node. Each training run took 4-6 hours with 72 GPUs.

Reports and presentations

Project Bibliography

  1. L. Ye, W. E. Allen, K. R. Thompson, Q. Tian, B. Hsueh, C. Ramakrishnan, A.-C. Wang, J. H. Jennings, A. Adhikari, C. H. Halpern, I. B. Witten, A. L. Barth, L. Luo, J. A. Mcnab, and K. Deisseroth, “Wiring and Molecular Features of Prefrontal Ensembles Representing Distinct Experiences,” Cell, vol. 165, no. 7, pp. 1776–1788, 2016.
  2. Karl Deisseroth, “Building a See-through Brain- A New Experimental Approach at the Interface of Chemistry and Biology,” Scientific American, 2016
  3. Q. Dou, H. Chen, L. Yu, L. Zhao, J. Qin, D. Wang, V. C. Mok, L. Shi, and P.-A. Heng, “Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1182–1195, 2016.
  4. C. Magliaro, A. L. Callara, N. Vanello, and A. Ahluwalia, “A Manual Segmentation Tool for Three-Dimensional Neuron Datasets,” Frontiers in Neuroinformatics, vol. 11, 2017.
  5. “NIH Launches the Human Connectome Project to Unravel the Brains Connections,” PsycEXTRA Dataset, 2009.
  6. N. J. Tustison, B. B. Avantis, P. A. Cook, Y. Zheng, A. Egan, P. A. Yushkevich, and J. C. Gee, “N4ITK: Improved N3 Bias Correction”, IEEE Transactions on Medical Imaging, June 2010, 29(6), pp. 1310-1320.
  7. Vikram Chandrasekhar, This method was developed in Dr. Sulam’s lab and based on template-matching.

Other Resources and Project Files

https://github.com/prernasingh11/CLARITY