Table of Contents

Photoacoustic Image Based Intra-Operative Surgical Guidance System in a da Vinci Surgical Robot Platform

Last updated: 03/31/2022

Summary

Interventional guidance systems are becoming increasingly common in modern surgical procedures to track the surgical tools. In nerve-sparing radical prostatectomy, surgeons need to track their surgical tools to avoid the intro-operation trauma of nerves and minimize the postoperative complication. We are implementing a Photoacoustic Image-Based Intro-Operative Surgical Guidance System in a da Vinci Surgical Robot Platform to assist in the surgical procedure. This guidance system depends on the registration of the ultrasound coordinate system with respect to the endoscopic camera coordinate system. Our responsibility mainly consists of three components: integration of the whole system, implementation and validation of the novel registration algorithm, and system optimization by introducing a photoacoustic (PA) visual servoing component. The goal of this project is to realize a fully automatic surgical tracking system to optimize surgical procedures.

Background, Specific Aims, and Significance

For patients with clinically localized prostate cancer, nerve-sparing robotic prostatectomy provides patients with a safe and minimally invasive technique for removal of the prostate gland, while preserving as much of the surrounding nerve structures responsible for penile erections. The advantage of this robotics-assisted method includes smaller incisions, reduced pain, blood loss, transfusion rates, and hospital stay, as compared to conventional open surgery with a similar cure rate. Driven by this, we here aim to develop a photoacoustic image-based intraoperative guidance system for da Vinci surgical robot. Specifically, this goal can be decomposed into three subtasks:
1. System integration: integrating all the components of the system with a GUI interface.
2. Implement a novel registration algorithm with the da Vinci robot platform. Compared with the conventional registration method, this novel algorithm can perform well with a simpler procedure.
3. Optimization by introducing photoacoustic visual-servoing. This module can automatically track the tooltip during surgery with little time delay.

Deliverables

Technical Approach

The whole system would be deployed upon da Vinci surgical robot platform, where we planed to implement three subtasks: system integration (minimum deliverable), a novel registration algorithm using virtual markers (expected deliverable), and photoacoustic visual servoing (maximum deliverable).
1. System Integration
We use PyQt as the frame to develop the GUI interface and ROS Melodic as the communication platform.
2. Novel Registration Algorithm
This subtask aims to implement a real-time ultrasound-camera registration by using photoacoustic virtual markers. A laser diode is attached to the surgical tool and capable of generating a photoacoustic point source which will be detected by the TRUS transducer. A novel algorithm in which a laser line and the geometrical loci (arc) from photoacoustic reception are registered each other to enable TRUS to track the surgical tooltip.
This novel registration algorithm does not require the visible laser spots in the endoscopic camera image. Instead, the algorithm depends on the pose of the laser diode from the endoscopic camera image. We plan to attach an Aruco marker to the diode and calibrate the transformation of the marker with respect to the diode.
The algorithm development (MATLAB code) and simulation study have been completed, thus our work is focusing on collecting data from the real scenario and implementing the registration algorithm on a robot platform based on ROS and C++. Besides, we also planned to design a GUI interface for better interaction with surgeons.

Dependencies

Completed:

• Da Vinci surgical robot platform (including endoscopic camera)

• TRUS transducer & US DAQ & actuator

• laser implementation (diode & generator)

• Phantom or ex vivo tissue

Uncompleted:

• Aruco marker & marker attachment & pointer, deadline: 4/22

Milestones and Status

GUI & System Integration - Milestone name: Overall design and development of GUI

- Milestone name: Implementation of laser excitation control


Conventional Registration Algorithm - Milestone name: Simulation analysis

- Milestone name: Implementation and integration

- Milestone name: Verification in the da Vinci platform


Novel Registration Algorithm - Milestone name: Simulation analysis

- Milestone name: Marker tracking module

- Milestone name: Implementation and Integration

Reports and presentations

Project Bibliography

Cheng, A. (2017). Developing Ultrasound-Guided Intervention Technologies Enabled by Sensing Active Acoustic and Photoacoustic Point Sources (Doctoral dissertation, Johns Hopkins University).

Chaumette, F., & Hutchinson, S. (2006). Visual servo control. I. Basic approaches. IEEE Robotics & Automation Magazine, 13(4), 82-90.

Chatelain, P., Krupa, A., & Navab, N. (2015, May). Optimization of ultrasound image quality via visual servoing. In 2015 IEEE international conference on robotics and automation (ICRA) (pp. 5997-6002). IEEE.

Machkour, Z., Ortiz-Arroyo, D., & Durdevic, P. (2022). Classical and Deep Learning based Visual Servoing Systems: a Survey on State of the Art. Journal of Intelligent & Robotic Systems, 104(1), 1-27.

Alexis Cheng, Hyun Jae Kang, Haichong K. Zhang, Russell H. Taylor, Emad M. Boctor, “Ultrasound to video registration using a bi-plane transrectal probe with photoacoustic markers,” Proc. SPIE 9786, Medical Imaging 2016: Image-Guided Procedures, Robotic Interventions, and Modeling, 97860J (24 March 2016).
cis2_research_proposal_zijian_shuojue.pdf

Other Resources and Project Files

Here give the list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space (2022-01).