Last updated: Apr 26th, 2019

Standard X-ray imaging brings difficulty for surgeons to identify region of interest (ROI) features from anatomical clutter. Dual energy X-ray enables anatomical clutter reduction via material decomposition by utilizing the physical properties of X-ray formulations. The figure below illustrates the decomposition process. Traditional Dual Energy X-ray Absorptiometry (DEXA) system has been developed in analyzing bone density, fat tissue, etc. But the model is largely approximated and simplified, because the target ROI is usually large and not targeted to small region accuracy. Thus, we propose to re-design the algorithm and improve the accuracy on decomposition of injected cement during femoroplasty. Decomposed frames can be used to improve 3D reconstruction of the injected cement, to better understand the cement distribution.

• Students: Cong Gao
• Mentor(s): Mathias Unberath, Mehran Armand, Russell Taylor

Conventional X-ray imaging is not sufficient to characterize object precisely, especially in the aspect of density, material identity, volume thickness, 3D depth of the object, etc. It is then hard for the surgeons to identify Region of Interest(ROI) using X-rays with multiple material stacked intensities. Thus, there is a need to develop an efficient decomposition system that can separate multiple materials in projection domain.

This project targets at utilizing the physical formulation of X-ray including the material attenuation coefficient, material density, X-ray casting length and especially photon spectrum energy. Here is how this formula looks like,

First, we conducted analysis using traditional model-based method. In the case of two materials, we can then further simplify this formula by its parameter w and the desired unknowns T for each pixel u:

The photon spectrum can be simulated using the online SIEMENS opensource simulation software. Below is the simulation result for 60kV and 75kV stop energy spectrum

The material attenuation coefficient is also simulated using the NDE open source data.(https://www.nde-ed.org/EducationResources/CommunityCollege/Radiography/Physics/attenuationCoef.htm) We can see an obvious K-edge effect (coefficient jump as particular energy level).

For Real X-ray validation, we plan to use the nView system, which is a fast 3D reconstruction system using low dose X-ray projections. Because the X-ray projections and reconstruction have correspondence with this system, its 3D reconstruction data can be used to label material groundtruth for validation. We plan to use this nView system to conduct femoroplasty injection experiment and collect dual-energy dataset during cement injection process for validation of the proposed algorithm.

Below is the figure of real X-ray acquisition:

Preliminary Result

By solving the linear model, we get the least square solution, but because this model is non-constrained, the output is highly overfitted to this data, which leads to negative T(u) problem, violating the physical constraints. Then we added a non-negative constraint to this least square solution. Below is the output,

In order to conduct simulation study, we want to use the recently proposed X-ray simulation framework – DeepDRR, which can perform fast and realistic simulation of fluoroscopy and digital radiography from CT scans by using deep learning. We can also simulate X-ray projections with different energy levels. Another benefit by introducing DeepDRR is that it enables segmentation in 3D domain, including bone, soft tissue and air, which can be used to generate target decomposition projections as groundtruth images for training.

Simulation study will start from testing on two materials. Iodine is our first target, because it has an obvious K-edge jump in its photon attenuation at around 70keV (shown in Figure. 3), which is a very good feature to do dual-energy decomposition. We plan to first simulate bone injection cement inside the femur, which mainly composes of iodine, and try to decompose it from the other background materials. If it works well, we then plan to test on more complicated situations. The right figure is the CT data we are using for simulation.

Simulation Results

Here are some simulation results of DeepDRR from a pelvis CT scan. The high energy image is the original simulation. The soft tissue and bone are generated from the traditional method.

We propose to introduce deep learning to this prediction framework, by explicitly including the physical constraint in the estimation part. Figure presents the overall workflow of the proposed pipeline. The input is the dual-energy acquisitions. We first run the traditional constrained least square solution to get an initialization result T(u). Then we try to train a discriminator to model the probability of the current solution P(T|M). This trained model is then used as a loss function to solve the overall combined optimization objective. This method will include both model driven and learning based method, which will avoid overfitting to training data, but biased by the physical model.

We first tested the results on the synthetic validation dataset. The accuracy is reported as mean dice score compared with the groundtruth decomposition and the signal variation. Since the predictions all have background noise, we are using a threshold of 1.2 times mean response to filter the noise. Then we compared the iodine prediction mask with the groundtruth mask. The mean dice score for model based method is 0.268, while our learning based method reaches 0.791. We also calculated the mean standard deviation of the output iodine response. The model based method has mean STD of 0.0037, while our learning based method is 0.0028. So the learning based method is largely outperforming model based method in prediction accuracy and smoothness of response.

We then tested the model on the unseen real X-rays. The constrained least squared solution has very sparse and limited signals, which are mostly be- cause of the overfitting of the linear model. Our learning-based model yields much better result in the cement region. Fig. 8 presents the synthetic valida- tion result and the real X-ray cement decomposition result. We can clearly see that the model based version has stronger noise and bad background prediction in the bone region, while the learning based method has more smooth signals and flatten background, which is closer to the real solution. The real X-ray decomposition also shows the shape of the injection tunnel and the strong distribution at the center of the head, which corresponds the real experiment condition.

1. Project Proposal & kick-off
   • Planned Date: Mid Feb.
   • Expected Date: Feb. 28th
   • Status: Done
2. nView system bone injection experiment
   • Planned Date: End Feb.
   • Expected Date: Feb. 25th
   • Status: Done
3. Finish simulation software and simulation dataset
   • Planned Date: Mid March
   • Expected Date: March 10th
   • Status: Successfully run through X-ray simulation software using DeepDRR framework from femur CT data. Test traditional mathematical method to decompose soft tissue and bone.
4. Finish network design for 2 materials, get simulation result
   • Planned Date: End March
   • Expected Date: End April
   • Status: Designing architecture is working as expected.
5. Finish network design for multiple materials, get simulation result (Canceled)
   • Planned Date: N/A
   • Expected Date: N/A
   • Status: N/A
6. Test on real dataset, get validation result
   • Planned Date: End April
   • Expected Date: May 6th
   • Status: We have conducted ablation study of model based method and learning based method on real data and got encouraging results.

• Project Plan
  • proposal.pdf
  • presentation.pptx
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Project checkpoint
• Paper Seminar Presentations
  • Seminar Paper Critique Writeup
  • Seminar Paper
  • Seminar Paper Presentation
• Project Final Presentation
  • PDF of Poster
  • Teaser Presentation
• Project Final Report
  • PDF of Final Report
  • links to any appendices or other material

1. Mazess, R. B., Barden, H. S., Bisek, J. P., & Hanson, J. (1990). Dual-energy x-ray absorptiometry for total-body and regional bone-mineral and soft-tissue composition. The American journal of clinical nutrition, 51(6), 1106-1112.
2. Rebuffel, V., & Dinten, J. M. (2007). Dual-energy X-ray imaging: benefits and limits. Insight-non-destructive testing and condition monitoring, 49(10), 589-594.
3. Albarqouni, S., Fotouhi, J., & Navab, N. (2017, September). X-ray in-depth decomposition: Revealing the latent structures. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 444-452). Springer, Cham.
4. Lu, Y., Kowarschik, M., Huang, X., Xia, Y., Choi, J. H., Chen, S., … & Maier, A. (2018). A learning‐based material decomposition pipeline for multi‐energy x‐ray imaging. Medical physics.
5. Ding, Q., Niu, T., Zhang, X., & Long, Y. (2017). Image-domain multi-material decomposition for dual-energy CT based on correlation and sparsity of material images. arXiv preprint arXiv:1710.07028.
6. Atria, C., Last, L., Packard, N., & Noo, F. (2018, March). Cone beam tomosynthesis fluoroscopy: a new approach to 3D image guidance. In Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling (Vol. 10576, p. 105762V). International Society for Optics and Photonics.
7. Unberath, M., Zaech, J. N., Lee, S. C., Bier, B., Fotouhi, J., Armand, M., & Navab, N. (2018, September). Deepdrr–a catalyst for machine learning in fluoroscopy-guided procedures. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 98-106). Springer, Cham.
8. Adler, Jonas, and Ozan Öktem. “Learned primal-dual reconstruction.” IEEE transactions on medical imaging 37.6 (2018): 1322-1332.

Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space (2019-99).