Table of Contents

Extrapolation of Missing Craniofacial Skeletal Structure via Statistical Shape Models

Last updated: May 9, 2014; 22:46 EDT

Summary

Statistical Shape Models are the primary topic of this project, however several sub-topics include surface and volumetric deformable registration, volumetric segmentation, feature extraction, and surface extrapolation. The goal is to design and implement a method for extrapolating missing anatomical craniofacial skeletal structure with the use of a statistical shape model of the human cranium.

[Figure 1: Basic Processing Flow]

Background, Specific Aims, and Significance

We intend to apply the algorithms developed for this project to the field of craniofacial surgery. The procedure of interest is craniofacial transplantation, which is the process of transplanting a donor’s craniofacial soft tissue, and possibly bone structure, onto a patient that has been subject to some severe craniofacial deformation. The surgery aims to restore lost functionality to the patient, such as the ability to smell, speak, or eat solid food [6]. By allowing the patient to participate in society as a “normal” individual, the surgery may help alleviate psychosocial traumas developed by the patient upon their disfiguration [6]. Figure 2 shows preoperative and postoperative views of a transplant recipient. Once a potential donor has been identified, the decision to perform surgery must be made within a very short time frame (24-36 hours)[7]. Amongst other factors, the skeletal structure of the patient and donor is compared for compatibility via cephalometric measurements [7]. We propose an attempt to estimate the skeletal structure of the patient’s face prior to disfiguration/injury. Once this estimated structure is completed for the patient, it may be matched quickly for compatibility with the donor’s skeletal structure and provide additional insight related to the patient-donor compatibility. If this skeletal comparison is accurate and useful, the surgeons may reprioritize their available time with additional refinement of the surgical plan or evaluation of other compatibility issues. It should be noted that the estimation of the patient’s ideal skeletal structure is an attempt to maximize the aesthetic quality of the surgical result, and makes no guarantee regarding the postoperative biomechanics of the patient. It is plausible that higher aesthetic quality will imply “good” biomechanics, but this will need to be the topic of further study. By leveraging techniques developed in the forensic facial reconstruction community [4], combined with the estimate of the patient’s true craniofacial skeletal structure, it may also be possible to create a model reflecting patient’s true physical appearance. [Figure 2: A craniofacial transplant recipient. Left preoperative photograph, middle preop- erative CT, right postoperative photograph.]

Deliverables

Technical Approach

This project consists of the following high-level technical components:

Once an atlas of skeletal surfaces has been created, the patient's skeletal surface will be deformably registered to the atlas, yielding an estimate of the patient's surface without abnormal pathology. The region of the patient's skeletal surface containing the abnormalities will be replaced with the corresponding regions from the atlas estimate. Additional processing will be required to remove any “jagged edges,” or discontinuities, as a result of the replacement. Figure 1 depicts a high level overview of the proposed reconstruction algorithm. Further details for each component are described in this section.

For this project, manual segmentation of the skeletal structure of the cranium will be performed. Open source, freely available, tools such as MITK, ITK Snap, and 3D Slicer may be used for this purpose [14][15][12]. This will most likely result in a more accurate segmentation compared to an automated method, and also avoid the “black art” of segmentation parameter tuning.

The general method we propose for atlas creation consists of the following steps and is a derivation of that found in Chintalapani, et al. [2]:

  1. Given a collection of CT images (not including the patient, IP), choose one as the template, IT, one as a test image, ITest, and denote the remaining N images as: I1, I2, …, IN.
  2. Construct a set of geometrically aligned and topologically consistent set of meshes for images IT, I1, I2, …, IN, denote them MT, M1, M2, …, MN.
  3. Compute the mean mesh, M~, from MT, M1, M2, …, MN and perform Principle Component Analysis (PCA) on the displacement vectors between the corresponding vertices on each mesh and M~.
  4. Using the eigenvectors output by PCA as variational modes, evaluate the atlas' accuracy representing the test subject with respect to the number of modes utilized, NM, for NM in { 1, 2, …, N, N+1 }. This is commonly referred to as the “leave one out” method.
  5. Choose NM* as the number of the modes to use in the atlas, where NM* represents the starting point of “diminishing returns” with respect to accuracy gained with increasing number of modes.

This approach differs most noticeably from the approach in [2], in that we will utilize only surface based mesh structures and operate on shape information only. We anticipate adequate results, even without the use of image intensity values, since the bone density of the skull is not of immediate importance.

We are currently evaluating two distinct methods for completing step 2 (creating MT, M1, M2, …, MN) in the general atlas creation creation method. The first method is summarized in the following steps:

  1. For i in {1, 2, …, N}, perform a volumetric deformable registration from Ii to IT. The output is a displacement field from exact voxels in IT to sub-voxels in Ii, denote it Di.
  2. Segment IT and create its surface mesh, MT.
  3. For i in {1, 2, …, N}, utilize Di to deform MT to Mi'.
  4. For i in {1, 2, …, N}, perform a rigid registration from Mi' to MT to obtain Mi.

The second method under consideration is summarized as follows:

  1. Segment IT, I1, I2, …, IN and create surface meshes: MT, M1', M2', …, MN'.
  2. Manually identify the locations of (a priori determined) anatomical landmarks on each mesh: LT, L1, L2, …, LN.
  3. For i in {1, 2, …, N}, perform a surface feature-based deformable registration, Di, from Li to LT.
  4. For i in {1, 2, …, N}, apply Di to MT to obtain Mi.

We plan to utilize Matlab, ITK, 3D Slicer, and MeshLab for mesh creation and modification operations [11][9][12][3].

After initial development of an atlas creation pipeline and structural extrapolation technique, we hope to integrate an iterative “bootstrapping” mechanism into the atlas creation process. We plan on following a similar approach as [2], with the exception of using only surface data.

In order to simulate a patient with some missing, or deformed, skeletal anatomy we plan on using MeshLab [3] to perform a straightforward cutting, or cropping, of the skeletal surface. We intend to perform an initial estimate of the patient’s surface using the “Statistical Atlas-Based Extrapolation of CT Data” method described in [1]. We refer to the existing skeletal structure of the patient, with deformities or missing structure, as the “known” structure, and any estimates using the atlas as the “estimated” structure. Denote the known surface mesh of the patient as M; it may be partitioned into two regions as shown in Equation 1, with one region representing the deformed region of the patient (MTransplant) and the other representing the unchanged region of the patient (MKeep). We can then perform a deformable surface registration of M to the previously created atlas; this yields an estimate of the “normal” patient (M') in terms of the mean shape (v0) and the modes (vi for i = 1, …, NM) as shown in Equation 2. We intend to use the method developed as part of CIS I Programming Assignment 5 to perform this registration [8]. An analogous partitioning of M' may then be created to obtain the estimate of the patient’s missing structure, shown in Equation 3. Replacing MTransplant with M'Transplant in M yields, an estimate of the patient's true skeletal structure, M∗, as shown in Equation 4.

[Equation 1] [Equation 2] [Equation 3] [Equation 4]

In most cases M∗ is an insufficient estimate, due to a discontinuity on the edge joining MKeep with M'Transplant [1], therefore some “smoothing” process is desirable. We hope to build off of existing reconstructive techniques in forensic anthropology [13]. Most of these techniques utilize thin plate spline (TPS) interpolation/extrapolation. Figure 3 depicts a “toy” one-dimensional example we have created for creating a smooth surface from two discontinuous inputs. We used a cubic-spline for interpolation in this example, which is the one-dimensional analog to the TPS. Matlab or ITK may be utilized for TPS implementations [11][9]. The smooth extrapolation was not implemented this semester due to insufficient time. Given sufficient time, we would also like to simulate a more realistic trauma to the patient (as shown in the middle of figure 1), and evaluate it’s registration to the atlas, and subsequent surface reconstruction.

[Figure 3: A toy problem for reconstructing a surface smoothly using cubic splines. (a) the known surface and landmarks (blue) and the estimated surface and landmarks (red); note the discontinuity between the two surfaces. (b) the smoothly reconstructed surface ignoring landmarks in overlap regions. (c) zoomed version of one overlap region from (b). (d) the smoothly reconstructed surface using a weighted combination of landmarks from each surface in overlap regions. (e) zoomed version of one overlap region from (d).]

Additionally, we plan on developing a similarity metric between an estimated patient skeletal structure and a potential donor's skeletal structure. This could initially be completed via a rigid registration and a computation of the Euclidian distance from the donor's surface to the estimated patient's surface within the transplant region of interest.

A joint effort between Johns Hopkins Hospital and Johns Hopkins Applied Physics Laboratory (APL) is currently underway to utilize preoperative CT data of the patient and donor, along with intraoperative processing, to obtain superior surgical outcomes [5]. We would like to extend this reconstruction method to assist in the craniofacial surgical planning phase of the APL project. The estimated skeletal structure may be able to enhance the selection of cutting planes for a transplant with minimal skeletal discontinuities.

We are also interested in developing a future system architecture that would incorporate a shared atlas amongst several participating hospitals, and an automated ability for obtaining a patient's skeletal reconstruction. Opposed to our manual segmentation of the skeletal regions of each cranium, an automated method should be employed or developed. Recent segmentation techniques, such Krach's “Sheetless Segmentation,” should be able to provided sufficient results [10]. An automated method of re-bootstrapping the atlas will need to be designed as well, so that the atlas may represent a larger segment of the population. If patient-donor compatibility is a concern, then a system of matching potential donors to patients, as the donors become available, would be another component. This would be relevant for automatically notifying the patient's surgeon when an ideal match for the patient becomes available in a different geographic locale; this could allow the donor to be immediately transported to the patient's institution before it becomes inviable for transplant. This system architecture would depend on consistent surgical planning, execution, and postoperative procedures across all participating institutions.

Sample Results

[Figure 4: Mean Shape and Modes 1-3 of the Atlas] [Figure 5: Extrapolation example with multivariate Gaussian regression; extrapolated area highlighted in green] [Figure 6: A craniofacial transplant candidate. Top: current skeletal structure. Middle: Atlas reconstruction. Bottom: Heat map of residual surface errors in normal regions]

[Figure 7: A synthetic disfigurement]

[Figure 8: Mean vertex errors for 0-3 iterations of deformable registration bootstrapping]

Dependencies

Milestones and Status

Summary: As of May 9, 2014, we have met our minimum and expected deliverables. External Link

Detailed Schedule

Project Milestones Chart

Reports and presentations

Project Bibliography

Other Resources and Project Files