Last updated: May 9, 2020 4:00pm

As robotic surgical systems become more and more prominent in hospitals around the world, efficient training in using these systems becomes vital to maximizing the effectiveness of the surgeons and the machines. Our goal with this project is to develop a collateral control system in the AMBF simulator in order to allow for effective surgical training and puzzle completion between two users. Along with developing such a system, a user study will be conducted to test how well these systems fair with real users and surgical experts in instruction and manipulation efficiency.

• Students: Bryan Birthwright, Joao Kawase
• Mentor(s): Dr. Adnan Munawar Ph.D

Robotic surgical systems have become increasingly popular around the world as a tool for surgeons to be able to perform increasingly effective and minimally invasive procedures. Due to this lack of training and access to these machines alternatives for developing the necessary skills for the operation of these machines is crucial for the proliferation of their use, and improvement of their effectiveness as powerful surgical instruments. The aims of this project are:

1. Develop shared control systems for the AMBF -Asynchronous Multibody Framework- simulator (a program capable of simulating virtual environments and tools that are integrable with a dVrk)

2. Develop a new set of puzzles within the AMBF simulator in order to prove the robustness of the simulator as a potential training tool

3. Conduct user study to determine how users most effectively learn how to use the control console with the least amount of frustration and time consumption

The significance of this project is to improve the efficiency of widely distributed surgical training platforms in education and proficiency-building activities. This project is a first step in developing these new systems which could help pave the way for creating new standards in already existing -however not fully standardized- robotic training curricula.

• Minimum: Expected by 4/10
  1. 2 new Dual console control schemes
  2. 6 new AMBF puzzles
  3. Data acquisition script for the user study

• Expected: Expected by 5/1
  1. All previously mention deliverables
  2. Completed User Study

• Maximum: Expected by 5/13
  1. All previously mention deliverables
  2. An academic paper analyzing and discussing the results of the user study

The technical approach for this project is three-fold: development of dual console control schemes, creation of a set of puzzles in AMBF simulator, and the undertaking of a user study to test the effectiveness of these newly developed collateral control systems when compared to traditional learning methodologies. First, collateral control systems will be implemented in the AMBF simulator using C++, which will be added on to already existing, but rudimentary shared control schemes. Second, Blender™ will be used to create 6 new 3D puzzles of varying difficulty (two easy, two medium, two hard) to be then imported into AMBF for use in the user study. Finally, a user study with an estimated size of 30 participants along with 2 experts will be conducted to gather data on user performance after self-teaching versus user performance after collateral control instruction from an expert.

Edit: Due to Covid-19 and class cancellations the user study portion of this project was not completed. Instead all the minimum deliverables were met, which include the dual control system, puzzles and data acquisition script.

• Image outlining the 5 puzzles which were developed for this project

• Example of data to be acquired from the python script

Currently unresolved dependencies:

1. Homewood IRB approval to be able to conduct

• Needed By 4/17
• In Progress
• If not resolved user study will not be conducted and only minimum deliverable can be satisfied

2. J-Card Access to JHU Robotarium to have access to dVRK

• Needed by 3/6
• In Progress
• If not resolved our access to dVRK will be limited but not completely impeded

3. Subjects for the user study (Surgeons and trainees)

• Needed by 4/10
• Not Started
• If not resolved user study cannot be conducted and only minimum deliverable can be satisfied

4. Data collection script for gathering user study data

• Needed by 4/10
• In Progress
• If not resolved minimum user study data will be gathered and some will have to be done manually

Satisfied dependencies: 1. Blender software for designing 3D puzzles for the AMBF simulator

• Needed by 2/27
• RESOLVED

2. AMBF Simulator source code for coding collateral control schemes

• Needed by 2/27
• RESOLVED

3. Computer running macOS or Linux for using Blender and coding in C++

• Needed by 2/27
• RESOLVED

4. Repo for puzzles and shared control schemes

• Needed by 2/27
• RESOLVED

1. Milestone name: Submit Request for Homewood IRB approval
   • Planned Date: 3/17
   • Expected Date: 3/17
   • Status: In Progress
2. Milestone name: Completed Dual Console control in AMBF and Full set of puzzles
   • Planned Date: 3/27
   • Expected Date: 3/27
   • Status: In Progress
3. Milestone name: Coded and Tested Data aquisition script
   • Planned Date: 4/10
   • Expected Date: 4/10
   • Status: Not Started
4. Milestone name: Completed User Study
   • Planned Date: 5/01
   • Expected Date: 5/01
   • Status: Not Started
5. Milestone name: Final Report, Paper, and Poster
   • Planned Date: 5/5
   • Expected Date: 5/5
   • Status: Not Started
6. Milestone name: Finished User Study Paper
   • Planned Date: 5/13
   • Expected Date: 5/13
   • Status: Not Started

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentations
  • M. Shahbazi, S. F. Atashzar and R. V. Patel, “A dual-user teleoperated system with Virtual Fixtures for robotic surgical training,” 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, 2013, pp. 3639-3644. https://ieeexplore-ieee-org.proxy1.library.jhu.edu/document/6631088/authors#authors
  • Dual Tele-operated system with Virtual Fixtures Presentation
  • Dual Tele-operated system with Virtual Fixtures Critical Summary
  • K. Shamaei, L. H. Kim and A. M. Okamura, “Design and evaluation of a trilateral shared-control architecture for teleoperated training robots,” 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, 2015, pp. 4887-4893. https://ieeexplore-ieee-org.proxy1.library.jhu.edu/document/7319488
  • Design and Evaluation of a Trilateral Shared-Control Architecture Presentation
  • Design and Evaluation of a Trilateral Shared-Control Architecture Critical Summary
• Mini-Checkpoint Presentation
  • Mini-Checkpoint Presentation
• Project Final Presentation
  • PDF of Presentation
• Project Final Report
  • Final Report

Bric, Justin D, et al. “Current State of Virtual Reality Simulation in Robotic Surgery Training: a Review.” Surgical Endoscopy, U.S. National Library of Medicine, June 2016, www.ncbi.nlm.nih.gov/pubmed/26304107.

Bric, Justin, et al. “Proficiency Training on a Virtual Reality Robotic Surgical Skills Curriculum.” Surgical Endoscopy, U.S. National Library of Medicine, Dec. 2014, www.ncbi.nlm.nih.gov/pubmed/24946742.

Lerner, Michelle & Ayalew, Mikias & Peine, William & Sundaram, Chandru. (2010). Does Training on a Virtual Reality Robotic Simulator Improve Performance on the da Vinci (R) Surgical System?. Journal of endourology / Endourological Society. 24. 467-72. 10.1089/end.2009.0190.

Moit, Harley, et al. “A Standardized Robotic Training Curriculum in a General Surgery Program.” JSLS : Journal of the Society of Laparoendoscopic Surgeons, Society of Laparoendoscopic Surgeons, 2019, www.ncbi.nlm.nih.gov/pmc/articles/PMC6924504/.

Sridhar, Ashwin N, et al. “Training in Robotic Surgery-an Overview.” Current Urology Reports, Springer US, Aug. 2017, www.ncbi.nlm.nih.gov/pmc/articles/PMC5486586/.

Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space (456-2020-12).

Deliverables Repository: https://github.com/jkawase1/CIIS-ambf

AMBF Simulator Repository: https://github.com/WPI-AIM/ambf