Last updated: May 10 and 12 PM

This project aims at developing a mathematical model on surgical skill evaluation in endonasal surgeries to identify and model the motion of critical movements to determine when these movements can lead to surgical complications.

• Students: Vishwa Parekh
• Mentor(s): Dr. Gregory Hager, Daniel Mirota, Narges Ahmidi

Background:

• Endonasal skull base and sinus surgeries requires to operate in a region that is in vicinity of many critical regions like carotid artery and optic nerve.
• Thus this surgery involves a lot of critical movements on the surgeon's part which if not performed correctly can cause very serious injuries to the patient.

Specific Aims:

• Mathematically model the critical movements in the surgery
• Given a series of movements, determine whether this may lead to surgical complications or not.

Significance:

• This model will help train the surgeon better especially with the critical movements.
• With the help of this model, the surgeons will be able to determine, during the surgical procedure when the surgery is going well and when it is not and can take necessary precautions beforehand.

• Minimum: (Expected by March 31)
  1. Complete: Develop software to Record Data from the surgery
  2. Complete: Develop software to compute the various transformations using the tracker data
  3. Complete: Develop a software to compute Registration between camera motion and CT data using Tracker based as well as Video based registration

• Expected: (Expected by April 25)
  1. Minimum Deliverables
  2. Complete: Develop an algorithm to optimize registration over all frames and use it to optimize registration obtained using tracker based and video based registration.
  3. Complete: Model the different surgical movements for one surgeon.

• Maximum: (Expected by May 10)
  1. Expected Deliverables.
  2. In Progress Model the different surgical movements for multiple surgeons.
  3. In Progress Develop a model to classify a surgical movement into expert and novice category for different movements.

Figure 2 shows the summary of the technical approach to be followed in this project.

Figure 2. Summary of the technical approach for the project

RECORDING DATA FROM SURGERY

This step requires to write a software to record tracker data and the video from endoscope during an endonasal surgery. The system currently employed in the Operating Room is a tracker based navigation system is shown in Figure 3. The software will have a calibration mode and Surgery mode. Calibration mode will be enabled during camera calibration and surgery mode will be enabled during surgical procedure. The software will have an easy to use GUI to start/stop recording, set mode and will also display the progress of recording.

Figure 3: Diagrammatic representation of the system currently employed in the operating room

CAMERA MOTION - CT REGISTRATION (Solving for FCO, Figure 3)

This procedure involves three steps:

1. The preprocessing before registration calculation

2. Registration calculation.

3. Optimize the registration over all the frames.

The Pre Processing step:

The pre processing step involves writing a software to do the following tasks:

• Check the synchronization between video sequence and tracker data and remove any phase difference between them.
• We need to first calculate FTR, FTE, FCR, FEO in order to be able to calculate FCO
• FTR, TTE: These will be calculated by using the tracking data for the markers attached to the reference frame and the rigid body attached to the endoscope.
• FCR: This will be calculated using CT fiducials attached to the reference frame. An intrinsic transformation will need to be calculated between the Tracker markers and the CT fiducials to enable transforming data in tracker co ordinate to CT co ordinates.
• FEO :: Solve AX = XB in the figure below.

Registration calculation step

The next task is to register the camera motion to the CT (Calculating FCO). For this purpose we will use the data from the navigation system and the endoscopic video from the surgery to register the motion of the endoscope tip to the CT. The complete registration procedure is illustrated in the figure below:

Tracker Based Registration:

• Compute FCO=FCRFTR-1FTEFEO
• Problem: Generates an error of 2 mm.

Video CT Registration:

• Manually mark landmarks on an image and on the CT scan.
• Compute 2D-3D registration using the method in [2] to determine the 3D location of the camera in the CT coordinate system.

Optimize Camera-CT Registration

The result from tracker based registration produces a registration error of 2 mm while image based registration produces sub millimeter error, but is recorded only for a certain number of static frames. The aim of this step is to use the result from image based registration to optimize tracker based registration over all the frames. As shown in the figure 4 we use calibration to calibrate tracker based registration to produce minimum error.

Figure 4: Diagrammatic representation of optimization procedure. Figure cited from CIS 1 lecture slide on calibration

SURGICAL SKILL MODELING

• Manually segment the video frames into different movements (approach, handling the tool, etc.) for an expert conducted and a novice conducted surgery.
• Te feature vector that would characterize the task would be the location of the endoscope in the CT frame through the task
• Transform the coordinate frame from CT to an w.r.t. an anatomical landmark
• Train a Hidden Markov model for each of these tasks separately for an expert conducted surgery.
• Test the model for a dataset with unknown skill level.

• The complete procedure was tested on a recording conducted on a phantom
• Figure below shows the result from 2D-3D registration. The image on the left shows the camera pose before registration and the image on the right shows the camera pose after registration. The blue points indicate the position of the landmarks as segmented from CT and the red points indicate the position of the landmarks in the image coordinates

• After Registration Optimization, the change in the mean

error for 5 different static frames was as follows: Initial Error in each coordinate direction (in mm) = 2.1816 2.1214 1.1958 Total Error: 3.2695 mm Final Error in each coordinate direction (in mm) = 0.9283 1.2236 0.9067 Total Error: 1.7835 mm

• Figure here shows the motion plots for approach into the sinus cavity. The motion begins at the bottom of the plot and ends at the top of the plot

• CISST Library: Resolved
• Medtronic Stealthlink libraries: Resolved
• System with capture card to record data from OR: Resolved
• Training for Access to OR: Resolved
• Required system (Stealthlink Tracker, endoscope, Foot pedal) in the OR: Resolved, Mentors are working on to get this system into the OR
• Software for Camera Calibration: Resolved, Part of CISST library.
• Get access to the OR for recording a surgery: Pending, Dependent on when the surgeon has a surgery and whether the OR will be available for recording then.
• Software for Surgical Modeling: Pending, Available in the lab. Talk to mentors for access.

1. Milestone name: Recording Software
   • Planned Date: March 3
   • Expected Date: March 7
   • Status: Completed
2. Milestone name: Test the Recording Software in the Lab
   • Planned Date: March 17
   • Expected Date:March 19 April 16
   • Status: Completed
3. Milestone name: Install the software in the OR for recording a surgery during the week
   • Planned Date: March 19 - March 23
   • Expected Date: March 19 - March 23 April 23
   • Status: Completed
4. Milestone name: Registration Software
   • Planned Date: April 14
   • Expected Date: April 14
   • Status: Completed
5. Milestone name: Surgical Skill Modeling
   • Planned Date: May 5
   • Expected Date: May 5 May 31
   • Status: In Progress

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentations
  • Paper Seminar Presentation
  • Paper Seminar Review
• Project Final Presentation
  • PDF of Poster
• Project Final Report
  • Final Report

1. D. J. Mirota, H. Wang, R. H. Taylor, M. Ishii, G. L. Gallia, and G. D. Hager, “A system for video-based navigation for endoscopic endonasal skull base surgery,” medical imaging, IEEE transactions on, ISS. 99, 2011.
2. Chien-ping Lu, Gregory D. Hager and Eric Mjolsness, ‘fast and globally convergent pose estimation from video images’, IEEE transactions on pattern analysis and machine intelligence, 2000
3. Carol E. Reiley and Gregory D. Hager, 'Decomposition of Robotic Surgical Tasks: An analysis of Subtasks and Their Correlation to Skill.
4. D. J. Mirota, A. Uneri, S. Schafer, S. Nithiananthan, D. D. Reh, G. L. Gallia, R. H. Taylor, G. D. Hager, and J. H. Siewerdsen, “High-accuracy 3D image-based registration of endoscopic video to C-arm cone-beam CT for image-guided skull base surgery,” in Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling, 2011, p. 79640j-1.
5. R.Y. Tsai, An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL, pp. 364-374, 1986

Source: source.rar