Contact Us
CiiS Lab
Johns Hopkins University
112 Hackerman Hall
3400 N. Charles Street
Baltimore, MD 21218
Directions
Lab Director
Russell Taylor
127 Hackerman Hall
rht@jhu.edu
Last updated: Mar 23th, 2021
Magnetic Resonance Imaging (MRI) is commonly used for diagnosis of the osteonecrosis of the hip. Core decompression is a commonly used surgical method for the removal of the ostenecrotic tissue (dead bone) from the femoral head. In order to remove necrotic tissue in the femoral head, surgeons rely on preoperative MR scans for tool trajectory planning and take intraoperative X-ray shots to monitor the procedure. The aim of this project is to develop algorithms that allow us to register the planned paths from the preoperative MR images to intraoperative X-rays to assist surgeons during core decompression surgery.
Translating annotated paths in pre-operative MR to X-rays requires a robust registration method between MR images and X-ray images. Being able to visualize the drilling paths in intraoperative X-ray images would significantly help the surgeons during the core decompression surgery. Nevertheless, direct registration between MR images and X-ray images has been challenging due to the lack of structural information between the two modalities.
The aim of this project is to create a workflow that allows this registration process to be achieved. In this work, a two-step registration method is proposed: (1) Synthesization of CT images given MR images as inputs (2) Registration between synthesized CT images and target X-ray images. The algorithms developed for the two steps will be tested and evaluated using the AVN Dataset, which consists of medical images of different image modalities from 30 patients.
The currently available MR to X-ray registration algorithms are mostly intensity- or feature-based methods, which are likely to work well on specific datasets and may not produce satisfactory results when applied to images focusing on another body part. This project provides a learning-based solution for the registration task. If the proposed method is successfully developed, it would be a more flexible approach that could work well on most datasets compared to existing methods as most model parameters would be learned during the training process.
In order to perform MR to X-ray registration, the following two steps are performed. (Updated on Feb 24th)
A cycle-consistent adversarial networks (CycleGAN) \cite{zhu_park_isola_efros_2017} is an unsupervised image-to-image translation network. Unlike other image translation network architectures which require paired images as the training dataset, CycleGANs learn from unpaired images and perform image translation in the absence of paired examples.
Fig.2 below illustrates the cycleGAN network architecture that would be developed in the project. Using the unpaired CT images and MR images from our datasets, four models would be trained, including the generator that synthesizes CT images from given MR images, the generator that synthesizes MR images from CT, the discriminator of CT images, and the discriminator of CT images.
The goal for this step is to produce a reliable MR to CT generator that could successfully generate realistic CT images from MR images as shown in Fig.3 below.
Progress
- 3/23 Update:
The current result of the CT generator of the trained cycleGAN model is shown in the image below. Images in the first row are the input MR images and the images in the second row are the synthesized CT scans. (The CT scans are standardized during the pre-processing process so the values don't look exactly the same as original CT scans)
The second step of our workflow is considered as a 2D/3D registration process. In a 2D/3D registration problem, the objective is to find a rigid pose of 3D data so that it aligns with the target 2D image. Once the 3D CT images are generated from the CycleGAN network, we will then perform 2D/3D registration between the synthesized CT and our target X-ray image.
One of the most common approaches to performing 2D/3D registration is through the production of digitally reconstructed radiographs (DRRs) from CT images through ray-casting \cite{sherouse_novins_chaney_1990}. Since the DRR generation process simulates the X-ray capturing process, the registration task could be formulated as an optimization problem that attempts to find the ideal rigid CT poses ($\theta$) that produce DRRs with the highest closeness to the target X-ray image as shown in the equation below:
In this work, the optimization-based approach will be performed to tackle the 2D/3D registration between the synthesized CT and the X-rays. Methods such as multi-start strategies \cite{otake_wang_webster} could be implemented for better initial pose estimation to prevent the optimization process being trapped in local optima.
Status updated on Apr 27th, 2021.
Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space (2021-10).
