The main goal of this project is to improve and automate the statistical atlas building pipeline developed by Dr. Gouthami Chintalapani at the Johns Hopkins University. We will then build a statistical atlas of the knee using CT images and use this statistical atlas for post-operative evaluations of anterior cruciate ligament (ACL) surgeries.

• Students: Murat Bilgel, Ceylan Tanes
• Mentor(s): Dr. Taylor, Xin Kang (Ben)

Currently, statistical atlas building methods depend largely on manual segmentation. Manual segmentation requires a lot of time and anatomical knowledge. By minimizing the human factor through the implementation of a semi- or fully-automated segmentation algorithm, the segmentation process will be more standardized, less prone to human error and the required time will be significantly shortened. Automating the current statistical atlas building pipeline will make the atlas building process more accessible to those with little or no programming experience.

Statistical atlases have a wide range of applications, including monitoring disease progression, accounting for anatomical variation in large populations and surgical planning. An automated atlas building pipeline will facilitate and increase the use of statistical atlases in surgical planning and post-operative evaluations. In this project, the specific application of the knee atlas will be to estimate the bone tunnel locations after ACL surgeries.

Our specific aims are:

1. develop a semi- or fully-automated segmentation method
2. automate the atlas building pipeline
3. develop a 3D-3D model-based registration algorithm
4. build a statistical atlas of the human knee
5. estimate bone tunnel locations of ACL surgery patients

• Minimum:
  1. Automate the pipeline developed by Dr. Chintalapani
  2. Build a statistical atlas of the bone structures of the human knee
• Expected:
  1. Develop a semi-automated method for knee segmentation
  2. Prepare detailed documentation of the improved pipeline
  3. Estimate bone tunnel locations using post-operative X-rays of ACL surgery patients
• Maximum:
  1. Develop a fully-automated method for knee segmentation
  2. Develop a 3D-3D model-based registration algorithm

Before working on automating the atlas-building pipeline and the segmentation/registration process, it is necessary to understand each component of the current implementation and how they interact with each other. The first step will be to gather all the programs and scripts required for the pipeline, and to run the whole process using sample data. Having a statistical atlas of the knee constitutes the first milestone of the project.

The second step will be to replace Analyze. ITK-SNAP, which is free software, can be substituted for Analyze to perform the preprocessing and the segmentation of the training images. Analyze is also used for other pre-processing tasks for the training images in the current pipeline, and MATLAB scripts can be used instead for those purposes.

The third step is automating the segmentation of the knee so that it requires minimal user interaction, if not none. Obtaining a semi-automated segmentation method constitutes the second milestone of the project. We will evaluate the performance of the semi-automated segmentation method against that of the manual method by comparing the segmentation accuracies and the amount of time it takes per dataset to perform segmentation.

The fourth step is to write a shell script that will automatically guide the user through the various steps of statistical atlas building. A fully automated pipeline is the third milestone.

We will then build the atlas of the knee using the improved pipeline. As an extension of the project, we will use the obtained atlas to estimate the bone tunnel positions using the knee atlas and two post-operative X-ray images of patients who underwent ACL surgery.

The fifth step is to develop a model-based 3D-3D registration algorithm that can be used as an alternative to Mjolnir, the registration algorithm currently used in the pipeline. Depending on the quality of the statistical atlases produced by the pipeline leading up to this step, this algorithm might not be used in the bootstrapping portion of the atlas building.

1. Obtain preliminary atlas using the current pipeline and existing pelvis data
   • by February 25 Done!
2. Create a tetrahedral mesh of femur and tibia using the Hong Kong dataset
   • Planned date: March 27
   • Expected date: March 27 Done!
   • Status: Cadaver images have been split into left & right femur and tibia. Segmentation has been done using ITK-SNAP and the surface mesh obtained from this software has been used for tetrahedral mesh generation in MATLAB.
3. Automate the pipeline
   • Planned date: March 27
   • Expected date: March 27 Done!
   • Status: FANTASMscript.sh and MJOLNIRscript.sh, the shell scripts used to perform the registration step in the atlas-building pipeline, are rewritten to accept command line arguments. Access to the Stomach server through X11 has been established. The shell scripts have been integrated into a MATLAB script that will guide the user through each component of the registration and allow for visualization of the results.
4. Obtain knee atlas
   • Planned date: April 24
   • Expected date: April 30 Done!
   • Status: We completed the preprocessing of patient images. We wrote a MATLAB script to automatically crop or pad the images to ensure that they all have the same number of voxels. We made the voxels isotropic using trilinear interpolation. We ran FANTASM on all data sets. We also made a video that guides the user through the scripts used in the pipeline. We resolved our problem with MJOLNIR, which was due to a wrong coordinate system specification during the tetrahedral meshing process. In order to be able to run MJOLNIR without an out of memory error, we had to squash the images and the meshes in half. We obtained the statistical atlas of the left and right femur using 8 CT data sets.
5. Estimate bone tunnel locations after ACL surgery
   • Planned date: May 6
   • Expected date: –
   • Status: We created a surface mesh necessary for the 2D-3D registration algorithm form the mean volumetric mesh. We also converted the volumetric modes to surface modes. However, we were unable to complete this milestone due to time constraints.

• We got the IRB approval to use patient data but getting the actual data took longer than expected. Therefore we are using cadaver bones from Hong Kong to build out atlas.

* We obtained a pelvis atlas using the existing data and the original pipeline. Through this process, we learned how each component of the atlas-building pipeline functions and identified the parts of code that need to be changed.

* All MATLAB code has been moved to the Stomach server (Linux). In order to be able to visualize the results while using Linux, we use ssh with X11. Shell scripts in the pipeline have been changed to accept command line arguments. The registration part of the pipeline has been integrated with the use of a MATLAB script.

* We pre-processed the Hong Kong dataset (cadaver dry bone scans) and split the images into left femur, left tibia, right femur, and right tibia. We stored each of these images as raw unsigned char files. We used ITK-SNAP to segment the images and used the segmentation to generate a surface mesh. The surface mesh was used to obtain the tetrahedral mesh that will be used in the registration step of the atlas building pipeline.

* The atlas building pipeline components have been integrated using MATLAB. The flow chart below demonstrates the tetrahedral mesh formation process.

* Image preprocessing for FANTASM and MJOLNIR has been performed. This includes ensuring that all patient images have the same number of voxels by cropping or padding the images, and making each voxel isotropic through trilinear interpolation.

* FANTASM and MJOLNIR results have been obtained with the Hong Kong data.

* PCA has been run on the results. A mean mesh and modes have been generated.

* A surface mesh has been created from the volumetric mean mesh from PCA results.

* A flowchart of our final pipeline is presented below.

* Below is a superimposition of the mean right femur mesh (blue) and its first mode image (red). Note that the distal end of the femur is at the top. The first mode captures the variation in the length of the femur.

* Below are the superimpositions of the mean right femur mesh (blue) and its first 3 mode images (red). Each column corresponds to a different mode. The first row is obtained by adding 3σ*mode to the mean shape, and the second column is obtained by subtracting this value from the mean shape.

* Below are the superimpositions of the mean right tibia mesh (blue) and its first 3 mode images (red). See above for the explanation of the image.

* The mean of the maximum vertex-vertex error values across all the subjects are presented below. Using more modes of variation when estimating shapes results in more accurate approximations.

* The same evaluation was performed with leave-one-out validation, and the results are presented below.

* After obtaining atlases for both the left and right femur, we investigated whether it is necessary to create separate atlases for the left and right side by taking the mirror image of the right femur atlas and quantifying how similar it is to the left femur atlas.

* Superimposition of the left femur mean shape (red) and the mirror image of right femur (blue) is presented below. The maximum vertex-vertex distance was 8 mm, indicating that separate atlases for the left and right femur might not be necessary due to the similarity between these two anatomical structures.

• Project Plan
  • Project Proposal
  • Project Proposal Presentation
• Meeting Minutes
  • Meeting Minutes 1
  • Meeting Minutes 2
• Project Background Reading
  • Ceylan:
    • A variational framework for integrating segmentation and registration through active contours by Yezzi, A. et al.
    • Paper Presentation
    • Paper Review
  • Murat:
    • Statistical Atlases of Bone Anatomy by Chintalapani, G. et al.
    • Paper Presentation
    • Paper Review
• Project Checkpoint
  • Project Checkpoint Presentation
  • Mini Checkpoint Presentation
• Project Final Presentation
  • Project Poster Presentation
  • Poster Teaser Slide
• Project Final Report
  • Project Final Report

1. M.A. Baldwin, J.E. Langenderfer, P.J. Rullkoetter, and P.J. Laz. Development of subject-specific and statistical shape models of the knee using an efficient segmentation and mesh-morphing approach. Computer Methods and Programs in Biomedicine 97 (2010) 232–240.
2. G. Chintalapani. Statistical Atlases of Bone Anatomy and Their Applications. Diss. Johns Hopkins University, 2010.
3. G. Chintalapani, L.M. Ellingsen, O. Sadowsky, J.L. Prince, and R.H. Taylor. Statistical Atlases of Bone Anatomy: Construction, Iterative Improvement and Validation. MICCAI, 2007. Part I, LNCS 4791, pp. 499-506. N. Ayache, S. Ourselin, A. Maeder (editors).
4. F. Ding, W. K. Leow, S. Wang. Segmentation of 3D CT volume images using a single 2D atlas. Proc. ICCV Workshop on Computer Vision for Biomedical Image Applications (2005).
5. L.M. Ellingsen, G. Chintalapani, R.H. Taylor, and J.L. Prince. Robust deformable image registration using prior shape information for atlas to patient registration. Computerized Medical Imaging and Graphics 34 (2010) 79–90.
6. T. Heimann, H. Meinzer. Statistical shape models for 3D medical image segmentation: A review. Medical Image Analysis 13 (2009) 543-563.
7. S. Martin, V. Daanen, J. Troccaz. Atlas-based prostate segmentation using an hybrid registration. Int J Computer Assisted Radiology and Surgery (2008) 3: 485-492.
8. H. Seim, D. Kainmueller, H. Lamecker, M. Bindernagel, J. Malinowski, and S. Zachow. Model-based Auto-Segmentation of Knee Bones and Cartilage in MRI Data. http://www.diagnijmegen.nl/~bram/grandchallenge2010/215.pdf.
9. G. Unal, G. Slabaugh. Coupled PDEs for Non-Rigid Registration and Segmentation. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005).
10. A. Yezzi, L. Zoellei, T. Kapur. A variational framework for integrating segmentation and registration through active contours. Medical Image Analysis 7 (2003) 2: 171-85.

• ITK-SNAP
• iso2mesh package for MATLAB
• STL_Import.m: MATLAB function to import .stl files (surface mesh) into MATLAB as list or vertices and triangles