Last updated: March 1st

Enter a short narrative description here

• Students: Mingxu Liu
• Mentor(s): Ping-Cheng Ku, Alejandro Martin-Gomez

Osteonecrosis of the femoral head (ONFH) refers to the disease that bone cells in the hip joint are dead due to insufficient blood supply to the femoral head. It could lead to failure of the subchondral bone. ONFH affects especially young adults aged between 30 to 50, and the incidence rate of ONFH keeps increasing.

(Image from: https://orthoinfo.aaos.org/en/diseases--conditions/osteonecrosis-of-the-hip)

Core decompression is a common treatment to both prevent or delay the worsening of early-stage ONFH or as a step of Total Hip Arthroplasty (THA) surgery when ONFH develops to subchondral bone collapse. For the core decompression process, MRI shows a great advantage in pre-operative imaging and surgical planning due to its sensitivity to necrosis, but it's in short of imaging bones. Intraoperatively, only CT is applicable. On the contrary to MRI, CT shows more advantage on imaging bones; however, necrosis tissue is hard to distinguish.

(Image from: https://www.stlosm.com/core-decompression-avascular-necrosis-of-hip-orthopedics-sports-medicine-specialists-creve-coeur-missouri.html)

A promising method to exploit the strengths of these two medical image technologies is to register MRI and CT volume. With appropriate registration, the surgical planning performed on the MRI imaging can be mapped to the CT volume. However, to accurately register the volumes, precise segmentation of CT and MRI are firstly expected. This is specifically hard for MRI due to its insensitivity to bone anatomy and lower contrast. Currently, the segmentation of MRI is done manually by expertise, which normally takes up to hours of work.

In this work, benefiting from the development of deep learning in medical image processing, the automatic segmentation of MRI volume of core decompression related anatomic structures is to be achieved. With the previous work on CT segmentation and 3D-3D registration, automatic and end-to-end segmentation of MRI and CT volumes for core decompression procedures, followed by the registration between the segments, is to be developed.

• Minimum: (March 23th)
  1. A model that automatically segments the femur, pelvis, and necrotic tissue from MRI for core decompression procedure.
  2. Code management and project documentation(throughout the entire project).

• Expected: (April 20th)
  1. By accomplishing the minimum objective, models to automatically segment the femur and pelvis from CT for the core decompression procedure.
  2. Use the MRI and CT segments, the pipeline to manually register the MRI and CT.

• Maximum: (May 3rd)
  1. Interactive design of the segmentation and registration pipeline with 3D Slicer.
  2. By accomplishing the expected objective, models to automatically register the femur and pelvis (MRI - CT).

MRI Segmentation

In this project, the segmentation of MRI volume is firstly realized based on the baseline model — the nnU-Net. nnU-Net is desired to provide an appropriate segmentation model for the rest of the work in this project. Four networks are designed to develop: 2D U-Net, 3D U-Net, 3D Low-Resolution U-Net and 3D Cascade U-Net.

                                 Illustration of MRI segmentation result.

CT Segmentation

For the CT segmentation, the conventional threshold-based method is shown to work on CT femur segmentation, and will be used as the baseline method for this project. The threshold-based algorithm introduced by Krčah et al.[9] will be used as the baseline model for the project.

The improved model for CT segmentation, if necessary, is planned to be another 3D CNN model. The threshold-based segmentation results (with appropriate manual corrections if needed) will then be used as the ground truth annotations for the improved models. Theoretically, the 3D CNN model for CT segmentation can be modified on the basis of the MRI segmentation model.

Method on MRI-CT registration

In this project, a two-stage registration based on segments from MRI and CT are realized.

The semi-automatic (automatic) registration is realized using (1). use ANTs to provide an initial estimation of the transformation between MRI and CT;(2) use MMI-based registration to compute the final transformation. Using the MRI and CT volume segments, the registration algorithm will then compute the transformation between MRI and CT. Finally, the two segmentation models and the registration model will be integrated together to form an end-to-end system and, if possible, to be integrated with 3D Slicer software.

1. Milestone name: Automatic Segmentation of MRI
   • Planned Date: Feb 25th
   • Expected Date: April 10th
   • Status: 100% done. Due to large training time-consumption and possible updates in annotation, the milestone is postponed to April 10. Procedure is parallel with CT segmentation.
2. Milestone name: Automatic Segmentation of CT
   • Planned Date: Feb 25th
   • Expected Date: April 13th
   • Status: 100%
3. Milestone name: Registration of MRI-CT
   • Planned Date: Feb 25th
   • Expected Date: May 3rd
   • Status: 100%

Timeline is shown as follows:

}

• Project Plan
  • automatic_segmentation_of_anatomical_structures_for_core_decompression_procedures.pdf
  • cis_proposal_cd.pdf
• Project Background Reading
  • critical_reading.pdf.
• Project Checkpoint
  • automatic_segmentation_of_anatomical_structures_for_core_decompression_procedures-checkpoint.pdf
• Paper Seminar Presentations
  • background_presentation.pdf
  • segmentation_of_the_proximal_femur_from_mr_images_using_deep_convolutional_neural_networks.pdf|}}
  • fully_automatic_and_fast_segmentation_of_the_femur_bone_from_3d-ct_images.pdf
• Project Final Presentation
  • cis_poster_group28.png
• Project Final Report
  • Final Report
  • links to any appendices or other material

1. Brian B Avants, Nick Tustison, Gang Song, et al. Advanced normalization tools (ants). Insight j, 2(365):1–35, 2009.

2. Paul J. Besl and Neil D. McKay. Method for registration of 3-D shapes. In Paul S. Schenker, editor, Sensor Fusion IV: Control Paradigms and Data Structures, volume 1611, pages 586 – 606. International Society for Optics and Photonics, SPIE, 1992.

3. Pall A. Bjornsson, Benedikt Helgason, Halldor Palsson, Sigurdur Sigurdsson, Vilmundur Gudnason, and Lotta M. Ellingsen. Automated femur segmentation from computed tomography images using a deep neural network. Medical Imaging 2021: Biomedical Applications in Molecular, Structural, and Functional Imaging, Feb 2021.

4. Yaw Boachie-Adjei, Jia-Wei Kevin Ko, and Quanjun Cui. Total hip arthroplasty after failed operative treatment for osteonecrosis of the femoral head. Seminars in Arthroplasty, 19(4):267–273, 2008. The Complex Primary Hip.

5. Cem M Deniz, Siyuan Xiang, R Spencer Hallyburton, Arakua Welbeck, James S Babb, Stephen Honig, Kyunghyun Cho, and Gregory Chang. Segmentation of the proximal femur from mr images using deep convolutional neural networks. Scientific reports, 8(1):1–14, 2018.

6. Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Petersen, and Klaus H Maier-Hein. nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2):203–211, 2021.

7. Stefan Klein, Marius Staring, Keelin Murphy, Max A. Viergever, and Josien P. W. Pluim. ¡emphasis empha- sistype=”mono”¿elastix¡/emphasis¿: A toolbox for intensity-based medical image registration. IEEE Transactions on Medical Imaging, 29(1):196–205, 2010.

8. Alexander Kolesnikov, Alexey Dosovitskiy, Dirk Weissenborn, Georg Heigold, Jakob Uszkoreit, Lucas Beyer, Matthias Minderer, Mostafa Dehghani, Neil Houlsby, Sylvain Gelly, et al. An image is worth 16×16 words: Transformers for image recognition at scale. 2021.

9. Marcel Krˇcah, G ́abor Sz ́ekely, and R ́emi Blanc. Fully automatic and fast segmentation of the femur bone from 3d-ct images with no shape prior. In 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pages 2087–2090, 2011.

10. Konstantinos N. Malizos, Apostolos H. Karantanas, Sokratis E. Varitimidis, Zoe H. Dailiana, Konstantinos Bargio- tas, and Thomas Maris. Osteonecrosis of the femoral head: Etiology, imaging and treatment. European Journal of Radiology, 63(1):16–28, 2007. Hip Joint.

11. Daniel Petek, Didier Hannouche, and Domizio Suva. Osteonecrosis of the femoral head: pathophysiology and current concepts of treatment. EFORT Open Reviews, 4(3):85 – 97, 2019.

12. S. Pieper, M. Halle, and R. Kikinis. 3d slicer. In 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821), pages 632–635 Vol. 1, 2004.

13. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmenta- tion, 2015.

14. R.A. Zoroofi, Y. Sato, T. Sasama, T. Nishii, N. Sugano, K. Yonenobu, H. Yoshikawa, T. Ochi, and S. Tamura. Automated segmentation of acetabulum and femoral head from 3-d ct images. IEEE Transactions on Information Technology in Biomedicine, 7(4):329–343, 2003.

This project will provide a GitHub repository containing all source codes. The link to the repository is:

https://github.com/JERRY-LIUMX/AVN-Automatic-Segmentation

The link is currently only visible to the project member and mentors, and will be visible publicly by the end of the project.

If you need data access, please contact Alejandro Martin-Gomez at alejandro.martin@jhu.edu.