Last updated: 04/02/2020 04:00AM

The goal of this project is to perform user study to assess the degree of tremor reduction in robotic microlaryngeal surgical procedures on cadaveric phantoms.

• Students: Jasmine (Sue Min) Cho
• Mentor(s): Professor Taylor, Balazs Vagvolgyi, Dr. Francis Creighton, Dr. Deepa Galaiya, Dr. Ioan Lina

Previous studies using JHU REMS robot and Galen Robot for “Operation Game” on phantom, Microsurgical anastomosis, and Laryngeal surgery have shown that the effects of hand tremor are eliminated when these robotic systems were used. However, quantitative assessment of tremor reduction is necessary in order to objectively compare the performance in free-hand surgery and robot-assisted surgery. With recently improved surgical microscope and video capture capability, we will be able to collect higher quality videos which will allow more accurate tremor reduction assessment.

1. Aim 1: Develop/adapt surgical tool tracking software using microscope video (with colored instruments)
   • Significance:
     • Facilitate automatic acquisition of instrument tracking data
2. Aim 2: Conduct user study & reduce data
   • Significance:
     • Provide meaningful data for analysis
3. Aim 3: Write paper with surgeons
   • Significance:
     • Share quantitative assessment results of tremor reduction in robotic microsurgical procedures with the medical and engineering community
     • Provide assessment for usefulness for tremor-eliminating robots (e.g. Galen) and may provide additional input for future design

• Minimum:
  1. Experimental apparatus (Expected by 03/15/2020)
  2. Documented code for surgical tool tracking software (Expected by 03/25/2020)

• Expected:
  1. Experimental data (Expected by 04/07/2020)
  2. Documented code for tremor reduction assessment (Expected by 04/07/2020)
  3. Report (Expected by 04/30/2020)

• Maximum:
  1. Academic paper (Expected by 05/13/2020)

1. Experimental apparatus
   1. Paint instruments with easily distinguishable colored nail polish + matte topcoat
   2. Use stereo camera → due to COVID-19 situation, monocular videos were taken

1. Surgical Tool Tracking Software
   1. Develop/adapt surgical tool tracking software using OpenCV
2. Data analysis
   1. Perform economy of motion analysis
   2. Perform frequency analysis

1. Access to computer
   • Proposed solution: Use personal laptop and back up on external hard drive
   • Alternative solution: Use spare personal laptop
   • Effect on milestone/deliverable: Needed for entire project
   • Needed by: Immediately
   • Status: Resolved
2. Access to OpenCV
   • Proposed solution: Use OpenCV that is already installed on my computer
   • Alternative solution: Use different opensource computer vision packages/libraries
   • Effect on milestone/deliverable: Needed for surgical tool tracking software
   • Needed by: Immediately
   • Status: Resolved
3. Video acquisition software (e.g. frame grabber)
   • Proposed solution: Coordinate with Balazs and Prof. Taylor to acquire necessary software
   • Alternative solution: Use existing data
   • Effect on milestone/deliverable: Needed for microscope video acquisition
   • Needed by: 03/15/2020 but as soon as possible for trial videos
   • Status: Resolved
4. Second Camera for stereo and adapter
   • Proposed solution: Coordinate with Balazs and Prof. Taylor to buy camera and adapter from Haag-Streit
   • Alternative solution: Use monocular image processing
   • Effect on milestone/deliverable: Needed for stereo image processing
   • Needed by: 03/15/2020 but as soon as possible for trial videos
   • Status: Resolved
5. Experimental protocol
   • Proposed solution: Coordinate with Prof. Taylor
   • Alternative solution: Use existing data
   • Effect on milestone/deliverable: Needed for user study
   • Needed by: 03/15/2020
   • Status: Resolved
6. Equipment (laryngeal tools, pig larynxes, nail polish, laryngoscopes, microscope, camera, Galen Robot)
   • Proposed solution: Coordinate with Prof. Taylor and surgeons, and order/gain access in advance
   • Alternative solution: Use existing data
   • Effect on milestone/deliverable: Needed for user study
   • Needed by: 03/15/2020
   • Status: Resolved
7. Surgical tool tracking software
   • Proposed solution: Consult with Balazs and develop software in time
   • Alternative solution: Focus more on adaptation than new development
   • Effect on milestone/deliverable: Needed to acquire instrument tracking data from videos
   • Needed by: 03/25/2020
   • Status: Resolved
8. Data – videos
   • Proposed solution: Make sure everything is ready on time
   • Alternative solution: Use existing data
   • Effect on milestone/deliverable: Needed as a new dataset for this project
   • Needed by: 03/31/2020
   • Status: Resolved
9. Availability of surgeons
   • Proposed solution: Work through Ioan and Deepa
   • Alternative solution: A) Use old data B) Enlist graduate students
   • Effect on milestone/deliverable: Needed for user study
   • Needed by: 03/15/2020
   • Status: Resolved
10. Tremor reduction assessment pipeline
    • Proposed solution: Develop pipeline by the end of user study
    • Alternative solution: Focus more on adaptation than new development
    • Effect on milestone/deliverable: Needed for assessing tremor reduction
    • Needed by: 04/07/2020
    • Status: Resolved

1. Milestone name: Completion of Surgical Tool Tracking Software Development/Adaptation
   • Planned Date: 03/15/2020
   • Expected Date: 03/15/2020
   • Status: Complete
2. Milestone name: Completion of Surgical Tool Tracking Software Documentation
   • Planned Date: 03/25/2020
   • Expected Date: 03/25/2020
   • Status: Complete
3. Milestone name: Completion of User Study and Data Collection
   • Planned Date: 03/31/2020
   • Expected Date: 03/31/2020
   • Status: Complete
4. Milestone name: Completion of Tremor Reduction Assessment Pipeline Development and Documentation
   • Planned Date: 04/07/2020
   • Expected Date: 04/07/2020
   • Status: Complete
5. Milestone name: Completion of Data Analysis
   • Planned Date: 04/19/2020
   • Expected Date: 04/19/2020
   • Status: Complete

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentations
  • Project seminar presentation
  • Project critical review
  • Selected paper
• Mini Checkpoint
  • Mini checkpoint presentation
• Project Final Presentation
  • PDF of Presentation
• Project Final Report
  • Final Report
  • https://drive.google.com/open?id=1_AckNOvGALFraMEArR5Kzc85FjVJfQZM

* here list references and reading material

Reading List

• Feng, Allen L., et al. “The Robotic ENT Microsurgery System: A Novel Robotic Platform for Microvascular Surgery.” The Laryngoscope, vol. 127, no. 11, 2017, pp. 2495–2500., doi:10.1002/lary.26667.
• Akst, Lee M., et al. “Robotic Microlaryngeal Phonosurgery: Testing of a ‘Steady-Hand’ Microsurgery Platform.” The Laryngoscope, vol. 128, no. 1, 2017, pp. 126–132., doi:10.1002/lary.26621.
• Ahmidi, Narges, et al. “Automated Objective Surgical Skill Assessment in the Operating Room from Unstructured Tool Motion in Septoplasty.” Johns Hopkins University, Springer Verlag, 8 Apr. 2016, jhu.pure.elsevier.com/en/publications/automated-objective-surgical-skill-assessment-in-the-operating-ro-3.
• Talukder, A., et al. “Real-Time Detection of Moving Objects in a Dynamic Scene from Moving Robotic Vehicles.” Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453), doi:10.1109/iros.2003.1248826.
• Verrelli, David I., et al. “Intraoperative Tremor in Surgeons and Trainees.” Interactive CardioVascular and Thoracic Surgery, vol. 23, no. 3, 2016, pp. 410–415., doi:10.1093/icvts/ivw150.
• Bouget, David & Allan, Max & Stoyanov, Danail & Jannin, Pierre. (2016). Vision-Based and Marker-Less Surgical Tool Detection and Tracking: a Review of the Literature. Medical Image Analysis. 35. 10.1016/j.media.2016.09.003.
• Maguire, Sharon K., et al. “Three-Dimensional Printing of a Low-Cost, High-Fidelity Laryngeal Dissection Station.” The Laryngoscope, vol. 128, no. 4, 2017, pp. 944–947., doi:10.1002/lary.26905.
• Razavi, Christopher R., et al. “Real-Time Robotic Airway Measurement: An Additional Benefit of a Novel Steady-Hand Robotic Platform.” The Laryngoscope, vol. 129, no. 2, 2018, pp. 324–329., doi:10.1002/lary.27435.
• Sznitman, R., Richa, R., Taylor, R. H., Jedynak, B., & Hager, G. D. (2013). Unified detection and tracking of instruments during retinal microsurgery. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35, 1263–1273.

• Google Drive:
  • https://drive.google.com/open?id=1_AckNOvGALFraMEArR5Kzc85FjVJfQZM

Or

• OneDrive:
  • https://livejohnshopkins-my.sharepoint.com/:f:/r/personal/rtaylor2_jh_edu/Documents/Galen%20-%20JHU%20Internal/Projects/Tremor%20Reduction%20Assessment/CIS2_Jasmine?csf=1&web=1&e=fW1kbF