Last updated: 14:42 5/6/2019

Resting state functional magnetic resonance imaging (rsfMRI) provides valuable insight into the brain's functional organization, however labeling network components and organizing them into the correct hierarchical structure remains a challenge. My project looks to automate the hierarchical organizing and labeling of rsfMRI networks components for diagnostics and pre-operative care.

• Students: Arden Chew
• Mentor(s): Dr. Mathias Unberath and Dr. Haris Sair

Functional magnetic resonance imaging (fMRI) is a medical imaging modality which measures brain activity by detecting changes in blood flow signals. Traditional task-based fMRI correlates tasks with brain activation, allowing for directed neural mapping. Unlike task-based fMRI, rsfMRI allows for image mapping of regional network interactions while in a task-negative state. This added advantage enables rsfMRI to explore the brain's functional organization.

Independent component analysis (ICA) has proven successful in segmenting rsfMRI data into distinct network components, which are structurally aligned in a hierarchical order. Variation in clustering and thresholding, however, can cause network merging or fragmenting into sub-components, thereby making correct labeling and organizing quite difficult. Brain networks of near identical shape and similar location often appear at higher order ICA. Differentiating such similar networks relies on an understanding of which greater network each split network is a subcomponent of. Current manual methods of hierarchical labelling require a strong understanding of the complex hierarchical network of brain components, which only a handful of neuroradiologists in the United States possess.

In addition to providing diagnostic value in differentiating brain tumors, brain neoplasms, arteriovenous malformations and other vascular malformations, correct labeling of brain components and subcomponents is critical for pre-operative care. rsfMRI labeling can both improve pre-surgical mapping and provide precise outcome prediction analysis. By automating this labeling process, the standard of care for regions without access to world leading neuroradiologists can be greatly improved. Therefore, in this project I look to automate the both the labeling of these network components and the functional organizing of sub-components. This specifically manifests itself in the following ways:

1. Development of a trained computational model with precise and consistent binary classification of rsfMRI network components (noise vs not-noise) in healthy patients
2. Improvement of this computational model for multi-label classification within a single group ICA analysis study, such that the number of network components is fixed (also in healthy patients only)
3. Finalization of this model to generalize well on all network components regardless of ICA network component parameters, and extrapolation onto patients with brain lesions

• Minimum: (3/15/19)
  1. A trained pytorch model file (.pt) for binary classification of rsfMRI network components in healthy patients
  2. A custom pytorch based network architecture file for aforementioned model and associated packages for data handling, training, and testing with complete documentation
  3. Clinically actionable testing data accuracy report (complete with testing and validation accuracies, confusion matrix of component labels, loss charting)

• Expected: (4/8/19)
  1. A trained pytorch model file (.pt) for multi-label classification of rsfMRI network components in healthy patients
  2. A custom pytorch based network architecture file for aforementioned model and associated packages for data handling, training, and testing with complete documentation
  3. Clinically actionable testing data accuracy report (complete with testing and validation accuracies, confusion matrix of component labels, loss charting)

• Maximum: (5/6/19)
  1. Drafted abstract/journal report of non-hierarchical multi-label classification of rsfMRI network component
  2. Python script for pipelining noise filtering into multi-label classification
  3. Python script for Pearson correlation thresholding and incorporation into full work-flow
  4. Clinically actionable testing data accuracy report

Data:
Over 1000 rsfMRI scans of healthy patients are publicly available on OpenNeuro. Furthermore, Dr. Haris Sair (my mentor) has provided me with binary noise labels on a group ICA of the Washington University School of Medicine 120 patient dataset. For initial network testing, I will reserve 40 rsfMRI scans for testing, while performing cross-validation on the remaining 80 scans. Given that 80 datapoints is not a particularly large amount for deep learning network classification, cross-validation will enable me to validate my data in a data-efficient manner while preserving the integrity of my test dataset. Once success is observed in testing of the initial dataset in binary and multi-class labeling of healthy patients, I will proceed to testing on a broader dataset of healthy patient rsfMRI scans across multiple hospitals. It is important to note that rsfMRI scans of patients with brain lesions are not publicly available, however Dr. Sair has completed at study of 100 subjects at Johns Hopkins Hospital which he will provide me access to once success is observed in the classification of network components in healthy individuals.

Model:
The primary network architecture approach will be a convolutional deep belief network.

The convolutional nature of this network architecture will allow for handling of volumetric BOLD signals from the rsfMRI scans. These scans are of pixel density 64x64x32 and normalized by temporal waveform. Given that I will be using an NVIDIA Titan GPU, I can reliably execute batching of this data in up to four samples in parallel.

The deep belief aspect of this network enables an unsupervised approach to identification of brain component hierarchy, while simultaneously incorporating this learned hierarchical knowledge into classification decisions. As depicted in figure 1, a deep belief network incorporates probabilistic pooling nodes known as restricted Boltzmann machines (RBMs). These undirected RBMs are latently ingrained and can establish connections with neighboring RBMs over training iterations, eventually mimicking a hierarchical structure. In a convolutional deep belief network, convolutional RBMs are stacked together. The standard loss function for deep belief networks which I will be using is contrastive loss. For optimization I will employ contrastive divergence which has been shown to work well in convolutional applications of deep belief networks, and I will tune a momentum parameter in my optimizer for improved functionality in the rsfMRI classification space.

Alternative:
The field of deep belief networks and convolutional deep belief networks remains relatively new. If I am unable to find success with this initial architecture, an ensemble approach will be considered. Standard convolutional neural networks have already been proven successful in classification of up to twelve rsfMRI network components through ICA. By ensembling these existing convolutional neural network architectures with a pure hierarchical model I may be able to correlate network structure to component labeling. Both random forests and agglomerative hierarchical clustering will be considered for ensembling application should the deep belief network fail.

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentations
  • Literature Review Report
  • Literature Review Presenataion
• Project Final Presentation
  • PDF of Poster
• Project Poster Teaser
  • Poster Teaser Slides
  • Poster Teaser Slides
• Project Final Report
  • Final Report

1. Hacker CD, Laumann TO, Szrama NP, Baldassarre A, Snyder AZ, Leuthardt EC, et al. Resting State Network Estimation in Individual Subjects. Neuroimage. 2013;15:616–633. 10.1016/j.neuroimage.2013.05.108
2. Shruti A, Sair H, Pillai J. Limitations of rsfMRI in the setting of focal brain lesions. Neuroimaging Clin N Am. 2017 Nov. 27(4); 645:661.
3. Bailey P, Zaca D, Basha M, Agarwal S, Gujar S, Sair H, Eng J, Pillai J. Presurgical fMRI and DTI for the Prediction of Perioperative Motor and Language Deficits in Primary or Metastatic Brain Lesions. Journal of NeuroImaging. 2015 July 14.
4. Hannawi Y, Lindquist M, Caffo B, Sair H, Stevens R. Resting brain activity in disorders of consciousness. Neurology. 2015 Mar. 84(12); 1272:1280.
5. Kulkarni T, Narasimhan K, Saeedi A, Tenenbaum J. Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation. Conference on Neural Information Processing Systems. 2016.
6. Salakhutdinov R, Tenenbaum J, Torralba A. Learning with Hierarchical-Deep Models. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2012 December 20. Volume 35; Issue 8; 1958:1971.
7. Lee H, Grosse R, Ranganath R, Ng A. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning. 2009 Jun 14. 609:616.

• Lee H, Grosse R, Ranganath R, Ng A. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning. 2009 Jun 14. 609:616.
• Lee H, Largman Y, Pham P, Ng A. Unsupervised feature learning for audio classification using convolutional deep belief networks. NIPS 2009.