======Objective Surgical Skill Assessment of Computer-Aided Hysterectomy Procedures====== **Last updated: Sunday, May 5, 2019 3:57 PM** ======Summary====== This is a project that strives to evaluate the skill level of the surgeon carrying out a hysterectomy procedure using the da Vinci Surgical System in the OR of Johns Hopkins Hospital. We have collected video footage as well as motion data from the robot to be used to solve the classification problem of "Expert vs. Trainee". * **Students:** Elif Bilgin (ebilgin1@jhu.edu) * **Mentor(s):** Anand Malpani, PhD (amalpan1@jhu.edu) & Molly O’Brien, BS (mobrie38@jhu.edu) * **Clinical Collaborator: ** Chi Chiung Grace Chen, MD MHS (Gynecology & Obstetrics) {{ :courses:456:2019:projects:456-2019-12:screenshot.jpg |}}
**Figure 1: Hysterectomy procedure, colpotomy step.** //(Image provided by mentors)//
---- ======Background, Specific Aims, and Significance====== **Background** Majority of previous research utilized virtual reality simulation, and captured data directly from the robot. These papers studied the following surgical tasks: * Suturing * Knot-tying * Needle passing * Cutting * Dissection, and used features including but not limited to: * Task completion time * Path length * Moving time * Velocity * Idle time * Energy activation Previous work from our lab focused on the final step of hysterectomy, the **vaginal cuff closure.** **Aim** Automatically assess skill in robot assisted hysterectomy procedures, particularly the **colpotomy** step, using video footage from procedures at Johns Hopkins Hospital, as well as motion data from the da Vinci Surgical System. **Significance** “Technological developments are enabling capture and analysis of larger amounts of complex surgical data…[This] allows for the objective computer-aided technical skill evaluation for scalable, accurate assessment; individualized feedback, and automated coaching.” ((Swaroop Vedula, S & O Malpani, Anand & Tao, Lingling & Chen, George & Gao, Yixin & Poddar, Piyush & Ahmidi, Narges & Paxton, Christopher & Vidal, René & Khudanpur, Sanjeev & Hager, Gregory & Chiung Grace Chen, Chi. (2016). Analysis of the Structure of Surgical Activity for a Suturing and Knot-Tying Task.)) Therefore, the ability to objectively assess the skill level of surgeons is critical for training future surgeons. **Key Terms** **Hysterectomy** is the process of the removal of the uterus. **Colpotomy** is a particular step in the hysterectomy procedure where the connective tissue attaching the uterus to the vaginal opening is removed to release the uterus before removal. ---- ======Deliverables====== * **Minimum:** (Expected by March 28th) - Statistics on video, kinematics and motion data, create dataset * **Expected:** (Expected by April 18th) - Apply feature-based ML methods to get an Expert/Trainee classification at an accuracy level of 70%. * **Maximum:** (Expected by May 2nd) - Apply Deep Learning methods to time series data such as DNNs and RNNs to get an Expert/Trainee classification at an accuracy level of 70%. ---- ======Technical Approach====== {{ :courses:456:2019:projects:456-2019-12:diagram_for_technical_approach.jpg?200 |}}
**Figure 2: Block diagram of technical plan composing of two phases (Click on image to enlarge)**
- ======Dependencies====== ^ Dependency ^ Status ^ Explanation ^ | IRB data access| Resolved |Have completed the proper training modules, have been added to the IRB study| |Access to the motion data which is under Intuitive NDA | Resolved |Signed the NDA | | Access to the Johns Hopkins University compute server | Resolved | Given proper credentials by mentors | |Obtain existing code for neural network methods from graduate student researchers | Resolved | Molly has agreed to sharing her implementation| ---- ======Milestones and Status ====== {{ :courses:456:2019:projects:456-2019-12:milestone.jpg?200 |}}
**Figure 3: Timeline diagram of milestones (Click on image to enlarge)**
- - Milestone name: Project Start * Planned Date: N/A * Expected Date: 31 Jan * Status: Completed - Milestone name: Project Plan Presentation * Planned Date: 14 Feb * Expected Date: 21 Feb * Status: Completed - Milestone name: Update Documentation * Planned Date: 14 Feb * Expected Date: 7 Mar * Status: Completed - Milestone name: Complete Minimum Deliverable * Planned Date: 14 Feb * Expected Date: 28 Mar * Status: Completed - Milestone name: Begin Expected Deliverable * Planned Date: 14 Feb * Expected Date: 30 Mar * Status: Completed - Milestone name: Checkpoint Presentation * Planned Date: 14 Feb * Expected Date: 11 Apr * Status: Completed - Milestone name: Complete Expected Deliverable * Planned Date: 14 Feb * Expected Date: 18 Apr * Status: Completed - Milestone name: Begin Maximum Deliverable * Planned Date: 14 Feb * Expected Date: 20 Apr * Status: Completed - Milestone name: Update Documentation * Planned Date: 14 Feb * Expected Date: 25 Apr * Status: Completed - Milestone name: Complete Maximum Deliverable * Planned Date: 14 Feb * Expected Date: 2 May * Status: Incomplete - Milestone name: Poster Presentation * Planned Date: 31 Jan * Expected Date: 9 May * Status: Completed ---- ======Reports and presentations====== * Project Plan * {{:courses:456:2019:projects:456-2019-12:cis_plan_presentation.pptx | Project Plan Presentation (Elif Bilgin - 2/19/2019)}} * {{:courses:456:2019:projects:456-2019-12:proposal.pdf|Project plan proposal}} * Project Background Reading * See Bibliography below for links. * Project Checkpoint * {{:courses:456:2019:projects:456-2019-12:elif_checkpoint_presentation.pdf| Project Checkpoint Presentation (Elif Bilgin - 4/11/2019)}} * Paper Seminar Presentations * {{:courses:456:2019:projects:456-2019-12:seminar_presentation.pptx|Seminar Presentation (Elif Bilgin - 3/14/2019)}} * {{:courses:456:2019:projects:456-2019-12:vedula.pdf|Seminar Paper}} * {{:courses:456:2019:projects:456-2019-12:elif_critical_review.pdf|Critical Review (Elif Bilgin - 3/13/2019)}} * Project Final Presentation * {{{{:courses:456:2019:projects:456-2019-12:elif_poster.pdf| PDF of Poster (5/8/2019)}} * {{:courses:456:2019:projects:456-2019-12:elif_poster_teaser.pptx| Teaser Slides (5/5/2019)}} * Project Final Report * {{:courses:456:2019:projects:456-2019-12:cis_ii_report.pdf|Final Report (5/8/2019)}} * links to any appendices or other material * Project Documentation * {{:courses:456:2019:projects:456-2019-12:documentation_cis.pdf|Project Documentation}} ---- ======Project Bibliography======= **Robotic Surgery Papers Regarding Hysterectomy** * Malpani, A & Martinez, N & Vedula, S & Hager, G & Chen, C. (2018). 17: Automated skill classification using time and motion efficiency metrics in vaginal cuff closure. American Journal of Obstetrics and Gynecology. 218. S891-S892. **Papers Regarding Scope of Project** * Azari, David P. et al. “Modeling Surgical Technical Skill Using Expert Assessment for Automated Computer Rating.” Annals of surgery (2017) * Ershad, Marzieh & Rege, R & Majewicz Fey, A. (2018). Meaningful Assessment of Robotic Surgical Style using the Wisdom of Crowds. International Journal of Computer Assisted Radiology and Surgery. 13. 10.1007/s11548-018-1738-2. * Swaroop Vedula, S & O Malpani, Anand & Tao, Lingling & Chen, George & Gao, Yixin & Poddar, Piyush & Ahmidi, Narges & Paxton, Christopher & Vidal, René & Khudanpur, Sanjeev & Hager, Gregory & Chiung Grace Chen, Chi. (2016). Analysis of the Structure of Surgical Activity for a Suturing and Knot-Tying Task. * Soto, Enrique & Lo, Yungtai & Friedman, Kathryn & Soto, Carlos & Nezhat, Farr & Chuang, Linus & Gretz, Herbert. (2011). Total laparoscopic hysterectomy versus da Vinci robotic hysterectomy: Is using the robot beneficial?. Journal of gynecologic oncology. 22. 253- 9. 10.3802/jgo.2011.22.4.253. * Poddar, Piyush & Ahmidi, Narges & Swaroop Vedula, S & Ishii, Lisa & Hager, Gregory & Ishii, Masaru. (2014). Automated Objective Surgical Skill Assessment in the Operating Room Using Unstructured Tool Motion. International journal of computer assisted radiology and surgery. * Zhang, Yetong & Law, Hei & Kim, Tae-Kyung & Miller, David & Montie, James & Deng, Jia & Ghani, Khurshid & the Michigan Urological Surgery Improvement Collaborative, for. (2018). Surgeon Technical Skill Assessment Using Computer Vision-Based Analysis. The Journal of Urology. **Machine Learning Papers** * Malpani, Anand & Swaroop Vedula, S & Chiung Grace Chen, Chi & Hager, Gregory. (2015). A study of crowdsourced segment-level surgical skill assessment using pairwise rankings. International journal of computer assisted radiology and surgery. * Zia, Aneeq & Essa, Irfan. (2017). Automated Surgical Skill Assessment in RMIS Training. International Journal of Computer Assisted Radiology and Surgery. 13. * Vedula, Satyanarayana S et al. “Objective Assessment of Surgical Technical Skill and Competency in the Operating Room.” Annual review of biomedical engineering 19 (2017): 301-325 . * Krishnan, Sanjay & Garg, Animesh & Patil, Sachin & Lea, Colin & Hager, Gregory & Abbeel, Pieter & Goldberg, Kenneth. (2017). Transition state clustering: Unsupervised surgical trajectory segmentation for robot learning. The International Journal of Robotics Research. ---- ======Other Resources and Project Files====== The official documentation can be found above, along with the final report and poster files.