Last updated: 5/10

Researchers have often relied on analysis of histopathology slides of the inner ear to understand the impact that different structures have on the various pathologies in otolaryngology.Currently, vascular disorders of the inner ear remain poorly understood. Some researchers have hypothesized that abnormal vasculature may play a role in pathologies such as vestibular neuritis.[3] The aim of this project is to create software for automatic segmentation and 3D reconstruction of vasculature within the inner ear to facilitate more efficient and spatial analysis of structures within inner ear

• Students: Aseem Jain
• Mentor(s): Francis Creighton, MD; Bryan Ward, MD; Amanda Lauer, PhD

Background

Researchers have often relied on analysis of histopathology slides of the inner ear from deceased humans/mammals to understand the impact that different structures have on the various pathologies in otolaryngology. Understanding the pathophysiology of the inner ear remains crucial to guiding treatment strategies in otolaryngology. For instance, the stria vascularis, a structure in the inner ear analyzed on histology slides, has been thought to be critical to understanding sensorineural hearing loss in mammals [1]. Additionally, through studying histology slides of patients with inner ear malformation, researchers were able to create a better model for cochlear implants in this population [2].

One area of interest within the realm of inner ear histopathology is the study of vascular in the inner ear diseases. Currently, vascular disorders of the inner ear remain poorly understood. Some researchers have hypothesized that abnormal vasculature may play a role in pathologies such as vestibular neuritis [3]. These researchers note that being able to analyze the vasculature with 3D models of histopathology could prove to be valuable as it would enable them to better understand the positional relationship between vasculature and adjacent structures.

However, despite the importance of analyzing histology slides, most researchers still rely on manual methods for segmentation of structures within the inner ear. Manual segmentation techniques can also lead to poor interrater reliability [4]. Furthermore, few researchers have the tools to be able to generate 3D models of structures within the inner ear. While automatic techniques to extract relevant features from whole slide histology images (WSI) is rapidly expanding, little work has been done to automate segmentation histology slides of the inner ear and construction 3D models from these segmentations.

Goals

The key deliverable at the end of this project is to create software that can

1. Use deep learning methods to segment histopathology slides of the ear
2. Use these segmentations to create a 3D reconstruction of the ear

These goals can further be organized into

• Minimum: Create software to perform semantic segmentation of the inner ear of macaque(monkey) ears
• Expected: Software that can segment and reconstruct a 3D volume of the inner ear of monkeys
• Maximum: Write and submit a manuscript that details proposed new software. Use software to analyze human ears: extract clinically relevant features of such as volume, density of cells, curvature, etc

Significance

Completion of this project will enable researchers to analyze WSI of the inner ear rapidly and accurately compared to existing manual techniques. Additionally, by adding the ability to view these segmentations in 3D, researchers will have new insights into the role that structures such as the vasculature can play in various pathologies. Creation of this software could also facilitate the ability to study other structures within inner ear WSI. The proposed project would be first example of reconstructing vasculature from inner ear histology slides using deep learning. Overall, the proposed project has the potential to dramatically improve the way researchers analyze WSI within the inner ear.

• Minimum: (Expected by 4/15)
  1. Scan and Label Appropriate structure from 1 monkey ear
  2. Implement U-Net with proposed algorithm
  3. Validation of U-Net on test data

• Expected: (Expected by 5/1)
  1. Align segmented images
  2. Reconstruction in 3D

• Maximum: (Expected by 5/20)
  1. Manuscript writing
  2. Analysis of human temporal bone

Broadly, the technical approach can be divided into two phases. 1) Creation of the deep learning framework for semantic segmentation of inner ear vasculature. 2) Alignment of segmentations and generation of 3D model

Phase 1

Data Acquisition: To create a deep learning framework, some WSI (40 images) of monkey ears must first be manually segmented. Segmented features will include inner ear vasculature, Scala media, Scala vestibuli and cranial nerve VIII.

Model Training: Like the approach outlined by Guo et al; various 300×300 patches will be acquired from the labeled data. Fine features will be extracted at the 20X resolution of the images while coarse features will be extracted from the 1X resolution. A U-net will be used initially to perform the semantic segmentation.

A 1X input image will be used initially to extract coarse/large features such as Scala vestibuli, CN VIII, etc. Once these features are extracted, a border will be created around them. Given that most vasculature is positionally nearby coarse features, the border will serve as a search boundary to extract and segment fine features from the 20X resolution. This method speeds up the processing time as only a few patches are analyzed along the search boundary. This method also addresses the challenge of extracting small features relative to coarse features. A similar approach was proposed in Guo et al for cancer segmentation.

Phase 2

Phase two of the project involves aligning and generating a 3D model of the inner ear vasculature. Segmented WSI from Phase 1 can be aligned with tools such as simple elastix. Open source software such as Slicer 3D can be used to render 2D segmentations into a 3D model.

This project requires digital WSI that will be provided by the Lauer Lab. Outside of this dependency most requirement of this project will be based on open source platforms. Other dependencies are outlined below

1. Milestone name: 40 labeled Whole Slide Images(WSI); 81 scanned slides total
   • Planned Date: 3/6/23
   • Expected Date: 4/4/13
   • Status: Completed
2. Milestone name: Functional software that segments vasculature from WSI
   • Planned Date: 3/16
   • Expected Date: 4/28
   • Status: In progress
3. Milestone name: Internal validation report that analyzes non-labeled slides
   • Planned Date: 3/23
   • Expected Date: 4/28
   • Status: In progress delayed
4. Milestone name: Software that can align WSI slides
   • Planned Date: 4/4
   • Expected Date: 4/4
   • Status: Completed
5. Milestone name: 3D mesh file of Monkey Inner Ear
   • Planned Date: 4/31
   • Expected Date: 4/31
   • Status: On track
6. Milestone name: Submitted manuscript
   • Planned Date: 5/11
   • Expected Date: 5/11
   • Status: In progress
7. Milestone name: 3D mesh file of Human Ear
   • Planned Date: 5/15
   • Expected Date: 5/15
   • Status: In progress

• Project Plan
  • Project plan powerpoint
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentation
  • Paper Seminar Presentation
• Paper Seminar Report
  • Paper Seminar Report
• Project Final Presentation
  • PDF of Poster
• Project Final Report
  • Final Report
  • links to any appendices or other material

1.Yu W, Zong S, Du P, Zhou P, Li H, Wang E, Xiao H. Role of the Stria Vascularis in the Pathogenesis of Sensorineural Hearing Loss: A Narrative Review. Front Neurosci. 2021 Nov 19;15:774585. doi: 10.3389/fnins.2021.774585. PMID: 34867173; PMCID: PMC8640081.

2.Monsanto RDC, Sennaroglu L, Uchiyama M, Sancak IG, Paparella MM, Cureoglu S. Histopathology of Inner Ear Malformations: Potential Pitfalls for Cochlear Implantation. Otol Neurotol. 2019 Sep;40(8):e839-e846. doi: 10.1097/MAO.0000000000002356. PMID: 31361687; PMCID: PMC7377297.

3.Büki B, Mair A, Pogson JM, Andresen NS, Ward BK. Three-Dimensional High-Resolution Temporal Bone Histopathology Identifies Areas of Vascular Vulnerability in the Inner Ear. Audiol Neurootol. 2022;27(3):249-259. doi: 10.1159/000521397. Epub 2021 Dec 29. PMID: 34965531; PMCID: PMC9133178.

4.Khened, M., Kori, A., Rajkumar, H. et al. A generalized deep learning framework for whole-slide image segmentation and analysis. Sci Rep 11, 11579 (2021). https://doi.org/10.1038/s41598-021-90444-8

5.Guo, Z., Liu, H., Ni, H. et al. A Fast and Refined Cancer Regions Segmentation Framework in Whole-slide Breast Pathological Images. Sci Rep 9, 882 (2019). https://doi.org/10.1038/s41598-018-37492-9

6.Jansen, I., Lucas, M., Savci-Heijink, C.D. et al. Three-dimensional histopathological reconstruction of bladder tumours. Diagn Pathol 14, 25 (2019). https://doi.org/10.1186/s13000-019-0803-7

Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space (2023-06).