Last updated: 1:00 pm, Feb. 18, 2020

Enter a short narrative description here

• Student: Yiping Zheng
• Mentor(s): Zhaoshuo Li, Russell Taylor
• Goal: Create a motion planning demo for Galen robot to perform robot-assisted mastoidectomy task

[P.H.: one photo or two]

Mastoidecotmy is a deliberate surgical procedure which is important to the treatment of diseases such as cochlear implant, acoustic neuroma etc. Surgeons have to mechanically drill a hole on patient's skull and all the way down to the meningeal, meanwhile carefully avoiding sensitive anatomy structure such as facial nurve, sigmoid sinus, and arteries. The drilling process is a challenging process to surgeons which often lasts 8 hours.

Surgical robots such as da Vinci Surgical System can mitigate the challenges by extending human capabilities. However, mainly because of the precision requirement of mastoidectomy is very high, so far there hasn't been any application or attempt of robot assisted mastoidectomy to our best knowledge.

Virtual fixture is software motion constraints, can further reduce the operational difficulties by allowing the surgeon and robot to work together to complete the surgical task with improved stability, reliability and precision. By tracking the relative position of the surgical tool with regard to patients body, it can stop the surgeon from making sudden motions and accidentally damaging critical anatomies and have the potentiality to make the robot assisted mastoidectomy possible.

Recently, a new virtual fixture generation algorithm was proposed which fits the scenario of mastoidectomy very well. By obtaining the 3D data of patient's skull, either pre-operatively via CT scan or intra-operatively via 3D ultrasound, it can generate virtual fixtures online from polygon mesh representations of complex anatomical structures and provide dynamic constraint formulation for the planning algorithm. The algorithm has been testified through validation and runtime experiments.

In this project, I'm going to integrate this anatomical virtual fixture generation algorithm with Galen surgical robot and perform a demo of robot-assisted mastoidectomy, which may be very useful to avoid touching patient's critical anatomy structure and mitigate surgeons' tension in the procedure.

Here we kept 2 versions of deliveralbes, idealistically original plan A and relistically revised plan B.

A. Original Plan

• Minimum: (Expected by Mar. 10, 2020)
  1. Simple geometry code integration

• Expected: (Expected by Apr. 17, 2020)
  1. Patient anatomy code integration(3D phantom)

• Maximum: (Expected by May. 1, 2020)
  1. Complete the user study.

B. Revised Plan under 2019-nCov

Since the laboratory access is not achievable any more, the goal for the CIS2 project has been shifted to simulation environment accordingly, and improving its robustness, rather than creating a demo in real world.

• Minimum: (Expected by Mar. 10, 2020)
  1. Get slack formulation integrated into the optimal controller.

• Expected: (Expected by Apr. 17, 2020)
  1. Perform the surgery task simulation with a simple robot model, with respect to the whole surface of the end-effector.

• Maximum: (Expected by May. 1, 2020)
  1. Perform the surgery task simulation with the Galen robot model, with respect to the complexity of its forward kinematics and Jacobian.

here describe the technical approach in sufficient detail so someone can understand what you are trying to do

The dependencies of the project is depicted in the following table. The items colored in red means not possible to achieve, green means has been achieved, yellow means yet to be achieved.

Phase 1: Getting Started, Mar.5 ~ Mar.19 (14 days)

1. Milestone name: Loading function for mesh STL binary file.
   • Planned Date: Mar.19
   • Expected Date: Mar.19
   • Status: 100%

Phase 2: Implement the Algorithm, Mar.20 ~ Apr. 5 (14 days)

1. Milestone name: Passing compilation of the integrated cotroller
   • Planned Date: Apr.5
   • Expected Date: Apr.5
   • Status: 90%

Phase 3: Test the controller with simple robot model, Apr.6 ~ Apr.20 (14 days)

1. Milestone name: A video demo of a simple robot that can actually perform the above tasks
   • Planned Date: Apr. 20
   • Expected Date: Apr. 20
   • Status: 0%

Phase 4: Test the controller using Galen robot model, Apr.21 ~ Apr.30 (10 days)

1. Milestone name: A video demo of the Galen robot that can actually perform the above tasks
   • Planned Date: Apr. 20
   • Expected Date: Apr. 20
   • Status: 0%

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentations
  • here provide links to all seminar presentations
• Project Final Presentation
  • PDF of Poster
• Project Final Report
  • Final Report
  • links to any appendices or other material

* here list references and reading material

1. S. Leonard, A. Sinha, A. Reiter, M. Ishii, G. L. Gallia, R. H. Taylor, et al. Evaluation and stability analysis of video-based navigation system for functional endoscopic sinus surgery on in vivo clinical data. 37(10):2185–2195, Oct. 2018

2. A. R. Widya, Y. Monno, K. Imahori, M. Okutomi, S. Suzuki, T. Gotoda, and K. Miki. 3D reconstruction of whole stomach from endoscope video using structure-from-motion. 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 3900– 3904, 2019 Abdomen SLAM

3. O. G. Grasa, E. Bernal, S. Casado, I. Gil, and J. Montiel. Visual slam for handheld monocular endoscope. IEEE transactions on medical imaging, 33(1):135–146, 2013.

4. N. Mahmoud, I. Cirauqui, A. Hostettler, C. Doignon, L. Soler, J. Marescaux, and J. M. M. Montiel. Orbslam-based endoscope tracking and 3d reconstruction. In CARE@MICCAI, 2016. Oral Cavity SLAM

5. L. Qiu and H. Ren. Endoscope navigation and 3D reconstruction of oral cavity by visual slam with mitigated data scarcity. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition Workshops, pages 2197–2204, 2018. ORB SLAM

6. Ra´ul Mur-Artal*, J. M. M. Montiel, Member, IEEE, and Juan D. Tard´os, ORB-SLAM: a Versatile and Accurate Monocular SLAM System. Member, IEEE, 2016

7. Xingtong Liu, Yiping Zheng, Russ Taylor, Mathias etc., Extremely Dense Point Correspondences in Multi-view Stereo using a Learned Feature Descriptor. CVPR 2020

Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space (456-2020-07).