Contact Us
CiiS Lab
Johns Hopkins University
112 Hackerman Hall
3400 N. Charles Street
Baltimore, MD 21218
Directions
Lab Director
Russell Taylor
127 Hackerman Hall
rht@jhu.edu
Last updated: May 5, 2021
Ever since the “Back to Sleep” campaign had parents putting their babies to sleep on their backs, the incidence of “Flat Head Syndrome” in infants has skyrocketed to become a pediatric epidemic. Currently, pediatricians merely use observation and a measuring tape to examine baby head shape when diagnosing skull deformity issues like deformational plagiocephaly/brachycephaly (DPB) and craniosynostosis. The subtlety of these conditions has led to up to 86% of DPB cases going unidentified and untreated in the US [1,2,4].
To prevent the expensive and intensive treatments required for skull deformities diagnosed after six months of age, this project aims to create a software pipeline for 3D reconstruction of baby heads. These 3D models would allow pediatricians to identify skull deformities with much more accuracy, efficiency, and ease. Quick and early detection is the key to preventing open surgery, helmet therapy, and physical therapy - a relief for doctors, families, and babies everywhere [6].
Before an infant is six months old, their skulls are easily deformed, which can result in a number of cranial defects. The most common, deformational plagiocephaly/brachycephaly (DPB) (also known as “Flat Head Syndrome”) is a non-synostotic condition that causes facial asymmetry. Craniosynostosis is another (less common) condition that is severe and highly morbid. There is currently no advanced technology that pediatricians and primary care providers can use to accurately diagnose these conditions. Instead, physicians simply use a measuring tape to measure head circumference, which is highly prone to error and misses the majority of cases [1]. While PediaMetrix has developed a computer vision algorithm for analyzing 2D images from above the baby's head, there is important diagnostic information that cannot be represented in only two dimensions. Other 3D scanners exist but are only available at great expense for neurosurgeons, plastic surgeons, and helmet clinics. A mobile application solution, on the other hand, would be easily accessible for general pediatricians and primary care physicians as well.
The aim of this project is to develop a software pipeline that can automate the diagnosis process for an infant younger than six months. By attaching a depth sensor to a compatible tablet with a mobile application, photos of a baby's head can be taken from several angles to generate a depth map. We intend to take this depth information and use simple registration and pose graph methods to generate a 3D reconstruction of the baby's skull. This provides physicians with a digital model that can be measured for standard skull shape parameters in both 2D and 3D, such as cranial index, cranial vault asymmetry, and head circumference.
If this project is successful, it will play a key role in PediaMetrix's ultimate pipeline for both generating the 3D model and analyzing it with machine learning for automatic diagnosis. The ultimate goal is to facilitate early detection of skull deformities and prevent unidentified and untreated cases of these conditions to persist past six months of age. This will reduce the need for physical therapy, helmet therapy, and open surgery. Catching more cases of severe conditions like craniosynostosis will also reduce pediatric morbidity rates [2,4].
See below for more details on each step of this workflow. In general, we begin with a baby head (doll or phantom), and we will use our depth sensor to collect data of the head from multiple angles, each in their own local coordinate systems/frames. Then, all of this data will go into our sequential ICP algorithm for 3D reconstruction and output a 3D model. Then, we can evaluate it against other reconstructions and ground-truth of the same object, and we can visualize our result with a mesh. The final white box in the schematic is not within the scope of this project, and it represents the direction that PediaMetrix intends to go with our project results. Ultimately PediaMetrix’s goal is to input 3D models into a machine learning algorithm to perform automatic diagnosis.
A depth sensor (Structure Sensor, Occipital) attached to an iPad is provided by PediaMetrix. The iPad has a custom application from Dr. Gueler installed on it, which allows the user to take color images (RGB) and depth images (D) that are automatically uploaded to an Amazon AWS workspace.
For testing our pipeline, we are provided with a baby doll that we can take images of. PediaMetrix also has a 3D-printed head phantom of a baby and its corresponding STL file, which contains the surface geometry information for the phantom. The iPad has an application from the company Occipital, which has its own software for high-fidelity 3D reconstruction.
A depth map is akin to a grayscale image that acts as a heat map for the spatial distance from camera to object. Each pixel is populated with this measured distance instead of a brightness value. To get the distance between adjacent pixels, we will need the intrinsic parameters (focal lengths and optical center) of our camera, which we are given. Using this information, we can recreate a 3D space [5].
Our pipeline is in Python and primarily employs the use of Open3D, a Python library for 3D data and image processing. This library has numerous methods for point cloud generation, registration, and visualization [7].
Our data should consist of several depth images (around 30) of a baby's head from all angles. By using the provided camera intrinsic parameters, we can use Open3D to generate 3D point clouds from these depth images. Our next step is to register these point clouds together such that they are in a common coordinate system.
The approach we have decided to take for this registration is a sequential Iterative Closest Point (ICP) pipeline. ICP is a commonly used algorithm for iteratively comparing two point clouds in order to find a rigid frame transformation (consisting of a rotation and translation) to convert from one to another. In our case, we are defining the world frame as the frame of the first point cloud we collect. Please refer to our “Project Plan Proposal” document in the “Reports and Presentations” section below for specific details on basic ICP methodology.
To accomplish a sequential ICP pipeline, we iterate over every image in order. This assumes that images were taken consecutively and do not jump back and forth between significantly different angles. For each image, we perform ICP to register it to the previous image and store the resulting transformation in a list. When we are done, we compose all of the transformations in reverse order to sequentially register frame n to frame n-1, and then to frame n-2, etc. until we reach frame 1. The reason we do not do a pairwise registration that registers every frame to every other frame is because of the general smoothness of a baby's head, which lacks rich and definitive features. We have observed that this registration scheme will result in all point clouds overlapping as much as possible, instead of overlapping slightly and gradually growing the reconstruction to cover all angles.
As mentioned above, Open3D has several functions relevant to ICP, which we are heavily relying on. For our limited trajectory, no special frameworks or data structures are necessary to store the final point clouds.
Once we have registered all of our point clouds to a single frame (frame 1), we can simply combine them into one point cloud. Then, to avoid redundancy, we can down-sample to a desired voxel size and generate a mesh using Open3D. This mesh will be the final visualization of the 3D reconstruction that we generate.
During evaluation, our “test points” will be the point cloud making up the ground-truth or Occipital reconstructed model, and we will register those to the same frame as our own output point cloud. ICP with identity initialization is less reliable here, because our result and the ground-truth frames could be wildly different. Thus, we are simply running our evaluation program with an initialization transformation that is manually found through simple composition of rotations and translations. As long as that initialization is close enough, ICP can be used to bridge the gap and generate the final registration for evaluation purposes.
In order to make our pipeline robust to baby motion, we also intend to collect data with our baby doll on a moving shaker. This shaker will have controllable movement speed settings, which we will adjust to simulate certain amounts of “fidgeting” from the baby while we collect data. Motion is expected to compromise the fidelity of the depth data as we increase the speed of motion or when we introduce more complex motion such as acceleration.
In general, we expect motion to present as increased noise in our data. We can potentially address this noise with common noise-reduction and smoothing algorithms, adapted to depth maps. If motion is minimal, our ICP approach should still hold, as we are cropping our data to only include the head, which is a rigid body. We are also not incorporating use of RGB data, which would otherwise cause difficulties with motion because the location of RGB features relative to each other would change.
This project was mainly hosted on a private GitHub repository that only David, Tara, and PediaMetrix have access to. We have provided our documentation for the pipeline.
