Trace:

Differences

This shows you the differences between two versions of the page.

--- courses:456:2023:projects:456-2023-06:project-06 [2023/05/11 06:08] – [Deliverables] ajain36
+++ courses:456:2023:projects:456-2023-06:project-06 [2023/05/11 06:28] (current) – [Milestones and Status] ajain36
@@ Line 58: / Line 58: @@
 Broadly, the technical approach can be divided into two phases. 1) Creation of the deep learning framework for semantic segmentation of inner ear vasculature. 2) Alignment of segmentations and generation of 3D model
-**Phase 1**
+**Overall Approach**
+{{ :courses:456:2023:projects:456-2023-06:figure1.png?600 |}}
+**Deep Learning Pipeline**
 {{ :courses:456:2023:projects:456-2023-06:fig1.jpg?400 |}}
-Data Acquisition: To create a deep learning framework, some WSI (40 images) of monkey ears must first be manually segmented. Segmented features will include inner ear vasculature, Scala media, Scala vestibuli and cranial nerve VIII.
+Data Base: Temporal bone slices of 4 macaque monkey ears have already been sectioned for this project. Each ear contains 110 slides spaced 10 micrometers apart. The ears themselves are assumed to have no pathologies associated with them. These slides were converted into digital images using an Olympus microscope setup in the Lauer lab at JHMI. As shown above, Two structures were specifically labeled for training: the vasculature in the mid-modiolus of the inner ear and the scala media, scala vestibuli, and scala tympani (these were combined and referred to as scalas). Of the 110 slides in the database, 30 were randomly sampled for labeling.
-Model Training: Like the approach outlined by Guo et al; various 300x300 patches will be acquired from the labeled data. Fine features will be extracted at the 20X resolution of the images while coarse features will be extracted from the 1X resolution. A U-net will be used initially to perform the semantic segmentation.
+Model Training: The training process for both structures was done within a Roboflow© environment. For the scalas, each slide and corresponding label was first compressed to 512x512 pixel image and then augmented by rotation from -15 to 15 degrees as well as magnified from 0-20% percent. Overall, 30 scalas were represented in 267 images. Data augmentation remains a valuable tool to increase deep learning generalizability when working with relatively small datasets. Instead of compressing the original image by ~20x, training for the vascular segmentation model involved tiling a ~4x compressed slide (2048x2048 pixel) into 512x512 patches before data augmentation. The vasculature in the inner ear is a relatively small structure and is often lost if represented in an overly compressed image. Overall, 1312 512x512 patches were used for vascular segmentation model.
-A 1X input image will be used initially to extract coarse/large features such as Scala vestibuli, CN VIII, etc.  Once these features are extracted, a border will be created around them. Given that most vasculature is positionally nearby coarse features, the border will serve as a search boundary to extract and segment fine features from the 20X resolution. This method speeds up the processing time as only a few patches are analyzed along the search boundary. This method also addresses the challenge of extracting small features relative to coarse features. A similar approach was proposed in Guo et al for cancer segmentation.
+{{ :courses:456:2023:projects:456-2023-06:112640704-87c73380-8e7c-11eb-9927-7928938e01b3.png?400 |}}
-**Phase 2**
+The nnUnet architecture has already been described in detail in other works.[7] A generic overview of the architecture is shown above. nnUnet has additional pre-processing steps including normalizing the intensity range of the image. The nnUnet was trained on 5 folds of data with a training, validation, testing split of 60-20-20. Each fold was trained for a maximum of 20-30 epochs. After training, the optimal weights of nnUnet based on the 5 folds were chosen and used for inference.
-Phase two of the project involves aligning and generating a 3D model of the inner ear vasculature. Segmented WSI from Phase 1 can be aligned with tools such as simple elastix. Open source software such as Slicer 3D can be used to render 2D segmentations into a 3D model.
+**Image Registration and 3D Reconstruction**
+{{ :courses:456:2023:projects:456-2023-06:figure3.png?400 |}}
+Once the deep learning pipeline produced binary masks, the image registration module aligned these masks. This done in Python using the SimpleElastix toolbox. The overall process of this module is outlined above. Broadly, a compressed (512 x 512px) WSI and its subsequent WSI were aligned by finding a transform T. This transform T was then applied to both the vascular mask as well as the scalas mask to align them to the previous slide. The transform T is represented by two separate processes. First, an affine transform T1 was computed between the two images (I1 and I2) to generate an initial aligned image I1a. Then, a rigid transform T2 is computed between I1a and I2. The final transform T can be represented as T1 x T2. The method for calculating each type of transform is a traditional iterative closest point algorithm implemented by SimpleElastix.
+Once the aligned images were produced, they were uploaded in 3D Slicer to produce a volume rendering. This was done through the SlicerMorph extension which enables users to upload a series of images as a volume stack.
@@ Line 90: / Line 100: @@
     * Planned Date: 3/23
     * Expected Date: 4/28
-    * Status: In progress delayed
+    * Status: Completed
   - Milestone name: Software that can align WSI slides
     * Planned Date: 4/4
@@ Line 97: / Line 107: @@
   - Milestone name: 3D mesh file of Monkey Inner Ear
     * Planned Date: 4/31
-    * Expected Date: 4/31
+    * Expected Date: 5/5
-    * Status: On track
+    * Status: Completed
   - Milestone name: Submitted manuscript
     * Planned Date: 5/11
-    * Expected Date: 5/11
+    * Expected Date: 5/29
     * Status: In progress
   - Milestone name: 3D mesh file of Human Ear
     * Planned Date: 5/15
-    * Expected Date: 5/15
+    * Expected Date: TBD
     * Status: In progress
@@ Line 122: / Line 132: @@
     * {{ :courses:456:2023:projects:456-2023-06:paper_seminar_report.pdf |Paper Seminar Report}}
   * Project Final Presentation
-    * {{:courses:456:2023:projects:456-2023-06:final_poster_pdf.pdf|PDF of Poster}}
+    * {{ :courses:456:2023:projects:456-2023-06:pdf_poster.pdf |PDF of Poster}}
   * Project Final Report
-    * {{:courses:456:2023:projects:456-2023-06:final_report.pdf|Final Report}}
+    * {{ :courses:456:2023:projects:456-2023-06:final_report.pdf |Final Report}}
-    * links to any appendices or other material
+    * https://github.com/2014ajain/Vasculature_Seg_Recon.git
 ======Project Bibliography=======
@@ Line 139: / Line 149: @@
 .Jansen, I., Lucas, M., Savci-Heijink, C.D. et al. Three-dimensional histopathological reconstruction of bladder tumours. Diagn Pathol 14, 25 (2019). https://doi.org/10.1186/s13000-019-0803-7
+.Isensee, F., Jaeger, P.F., Kohl, S.A.A. et al. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 18, 203–211 (2021). https://doi.org/10.1038/s41592-020-01008-z
 ======Other Resources and Project Files======
 Here give list of other project files (e.g., source code) associated with the project.  If these are online give a link to an appropriate external repository or to uploaded media files under this name space (2023-06).
+https://github.com/2014ajain/Vasculature_Seg_Recon.git

courses/456/2023/projects/456-2023-06/project-06.1683785331.txt.gz · Last modified: 2023/05/11 06:08 by ajain36