Last updated: 5/11/2018

We are using surgical tool motion data from septoplasty, a common surgery used to correct deviated septums, to work towards discovering the geometry of patients' septums. This is incredibly important as currently septoplasty operates in a “black box” with no way to see the patient's septum due to the small size of the nasal cavity. By gaining insight into the geometry of patients' septums post-operatively without having to conduct CT imaging, we will be able to eventually enable a host of downstream applications including surgical training, skill evaluation, and enhanced patient care/treatment.

In this project, we are developing a simulation dataset of septoplasty tool motion, developing a method to isolate on-surface points from raw data, and testing a registration algorithm on several types of simulation experiments we generate. We are working towards the ideal solution of an atlas-based deformable registration algorithm using real septoplasty surgical data.

• Students: Rohit Joshi, Manyu Sharma
• Mentor(s): Swaroop Vedula, Anand Malpani, Molly O'Brien, Ayushi Sinha

Septoplasty is a medical procedure conducted to correct a deviated septum, and is considered one of the most basic surgeries an ENT surgeon has to master. In fact, this surgical procedure is the third most common ear, nose and throat surgery performed. The most important and complicated step in the surgery is the mucosal flap elevation, where the surgeon uses a tool called the cottle elevator to elevate the mucosal lining from the underlying cartilage of the septum to straighten it. The images below show a before and after picture of what would happen to a patient who has gone through septoplasty, and also the cottle elevator tool that surgeons use.

Currently, due to the size and position of the septum, septoplasty surgeries are performed in a “black box,” where no one is able to see exactly what is happening during the surgery, not even the surgeon himself. Because of this, a way of visualizing the geometry of a patient’s septum without CT imaging could be immensely useful in many ways, including but not limited to surgical training, enhanced patient care, and refinements to septoplasty procedure. Our mentors believe an ideal solution would be a deformable registration algorithm that makes use of an atlas. Our project is focused on moving towards that end goal, while also being achievable in a semester.

There are five main components of this project:

1. Develop septum models and collect ideal simulation data
2. Develop a more accurate analog for septoplasty and collect new simulation data
3. Use support vector machines to isolate the on-surface points from the raw simulation data
4. Use deformable coherent point drift on both sets of simulation data
5. Use Haussdorff Distance and other metrics to compare the accuracy of CPD

• Minimum: (Expected by 4/8/18)
  1. Develop phantom models for simulation data collection COMPLETED
  2. Build a dataset using phantom models and Aurora tracking device COMPLETED
  3. Visualizations from using the coherent point drift on “ideal” simulation data COMPLETED
  4. Comparison of our CPD model with the actual model COMPLETED

• Expected: (Expected by 4/22/18)
  1. Enhance the model to simulate mucosal flap elevation using gelatin peeling COMPLETED
  2. Visualizations from using deformable ICP on gelatin peeling simulation data
  3. Visualizations from using CPD on gelatin peeling simulation data COMPLETED
  4. Comparison of the two algorithms with the expected output and with each other SEMI-COMPLETED

• Maximum: (Expected by 5/7/18)
  1. Processed surgical data from the R21 dataset with accompanying visualizations of deviated septums created from deformable ICP and CPD
  2. Metric measuring accuracy of the outputs of these models, most likely computed from comparing the two with each other COMPLETED

As you can see, we were able to complete almost all our deliverables. The main deliverable that we were not able to fully accomplish that we had originally planned to was a fully working implementation of deformable ICP. Because this algorithm involves statistics, including PCA modes, it was significantly more work than we had planned. In the end, we were not able to fully implement deformable ICP, which resulted in one deliverable not being completed, and another missing about half of what it needed. Since the main purpose of our project was to prove the concept of any deformable registration working with septoplasty tool motion data in order to move to an atlas based registration algorithm in the future, we feel that using only deformable coherent point drift still achieves our main goals.

We also did not achieve part of our maximum deliverable. While we began working with the clinical data, after spending some time trying to see if we could use any of our methods with the surgical data, we found that it would be nearly impossible. The surgical data was much harder to use than we initially expected. Most of the cases we looked at had data clouds with no underlying structure at all, solely strokes leaving and entering the nose. The inherent problems such as the non-rigidity of the actual septum, the possibility of patient head motion (although occasionally corrected for with head sensors), and missing frames, were all together too many confounding factors for us to be able to make any sense of before the project deadline.

To summarize, we accomplished all of our minimum deliverables, missed one of our expected deliverables, and missed one of our maximum deliverables.

We want to start out by first collecting ideal simulation data in a controlled environment. We will do this by 3D printing three septum models to mimic the different possible septum deformations: one in a straight line to represent a normal septum, one in a C-shape to represent a possible deviated septum and one in an S-shape to model another potential deformity. For the first set of data, we will simply run our tracked tooltip over the surface of these models and minimize any motions leaving the surface. By doing this, we will limit the data to only the model surface. Then, we will use the coherent point drift algorithm to see if we can reconstruct the non-deviated septum from the data collected. This should give good results since our data is very ideal. We will measure how accurate this is by comparing it with the actual point cloud and also using Haussdorff distance

After this, we will collect data that more closely mimics the various motions in different planes that is found in real septoplasty by running our gelatin peeling simulation. In this, we will let gelatin dry on the outside surface of our septum models and proceed to use our tracked tooltip to peel back the gelatin layer. Next, we will process the data to understand which of these data points lie on the actual surface of the model septum and which are formed from the removing motion. After this, we will also re-run our CPD algorithm on this new data and compare the results from this to the ideal model.

By using this technical approach, we will be able to show that registration algorithms can be used to address this problem while also working towards the ideal solution of a deformable registration algorithm that makes use of an atlas.

Clicking on the table below shows our dependencies and how we plan to resolve them if things do not go as expected.

Clicking on the chart below will show a table of our milestones and planned dates of achieving them. Green milestones are the minimum, yellow milestones are the expected, and red milestones are the maximum. Because of changes in our project, the ICP and surgical data related milestones were removed and no longer pursued by the end of the project. All other milestones were met.

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for important papers.
• Project Checkpoint
  • Checkpoint Presentation
• Paper Seminar Presentations
  • Rohit's Seminar Paper Presentation
  • Rohit's Seminar Paper Report
  • Rohit's Paper
  • Manyu's Seminar Paper Presentation
  • Manyu's Seminar Paper Report
  • Manyu's Seminar Paper
• Project Final Presentation
  • PDF of Poster
• Project Final Report
  • Final Report
  • Github Repository
• Documentation
  • Septum Tracking Documentation

• Videos
  • Affine CPD of ideal data
  • Affine CPD of gelatin peeling data
  • Visualization of ideal data
  • Visualization of gelatin peeling data
• Images
  • 
    • Registering our curved model to our S-shaped model
  • 
    • Registering our curved model to our flat model

1. Poddar, Piyush & Ahmidi, Narges & Swaroop Vedula, S & Ishii, Lisa & Hager, Gregory & Ishii, Masaru. (2014). Automated Objective Surgical Skill Assessment in the Operating Room Using Unstructured Tool Motion. International journal of computer assisted radiology and surgery.
2. Berger, Tagliasacchi & Seversky. (2014). State of the Art in Surface Reconstruction from Point Clouds. The Eurographics Association.
3. Bernard, Salamanca, Thunberg. (2017). Shape-aware surface reconstruction from sparse 3d point-clouds. Medical Image Analysis.
4. Wu, Murtha, Jaramaz. (2015). Construction of statistical shape atlases for bone structures based on two-level framework. International Conference on Computer Vision Theory.
5. Myronenko, Song, Carreira-Perpiñán. (2006). Non-rigid point set registration: Coherent Point Drift. Conference on Neural Information Processing Systems.

See links section for useful resources