Last updated: May. 18

Microsurgery presents an array of problems. For instance, current methodologies of Eye Surgery requires freehand manipulation of delicate structures without haptic feedback. On top of this limitation, Eye Surgeons experience involuntary hand tremor, fatigue, poor visual cues, and low tactile sensation. Moreover, Surgeon training is limited. There exists no transition between cadaver surgery and live human surgery. In this project, we focus on introducing new features to the Eye Robot that serve to not only further the development of solving these aforementioned problems, but to also provide a framework for a smoother training process for upcoming Eye Surgeons through the use of Telemanipulation and Telestration.

• Students: Robert Eisinger, Orhan Ozguner
• Mentor(s): Marcin Balicki

Retinal diseases are becoming a common problem in the world, and they are a leading cause for blindness. Since the eye is one of the most fragile organs in the body, it requires extreme care before, during and after the surgery. The most frequent indications for retinal surgery include retinal detachment, vitreous hemorrhage, macular puckers and holes and diabetic retinopathy. Success rates of the surgery highly depend on the surgical techniques and skills. Surgical challenges:

• Difficulty on manipulating 1-100um delicate structure
• Physiological hand tremor and lack of force feedback
• Inadequate visualization and tissue depth perception
• Miniature instrumentation
• Involuntary patient movement
• Fatigue from long operations
• Surgeon training

In order to address these challenges, The Johns Hopkins University CIIS lab has been developing a computer integrated Surgical Assistant Workstation (SAW). However there is no such system that is dedicated to train a new surgeon or allowing multiple surgeons to work cooperatively.

Telemanipulation

Telemanipulation means manipulating a robot using a master control system. It allows us to control a robot remotely. In our system we will be controlling the stead hand eye robot (slave) via Phantom omni (master). This incorporation will be able to be used for many purposes like educating new surgeons or multiple surgeon incorporation for a single surgery. Since there is no such system that can help a new surgeon to train himself, using telemanipulation, an experienced surgeon can keep eye on a new surgeon to make sure that he is doing everything in a proper way.

Telestration

Telestration means freehand drawing with 3D display. This system will be used for many purposes like intraoperative communication between surgeons, education new surgeons, defining virtual fixtures. In this case, while a surgeon is controlling the steady hand eye robot, another surgeon will be using Phantom Omni. While using the Omni, a surgeon will be able to draw an area on the 3D display, for communication or for defining virtual fixtures.

Currently there is a rough prototype for telemanipulation, however there is a problem on the rotations which leads the robot move to a undesired position. Also, there is an application for telestreoviewer and retina tracker. The specific aims of the project can be listed as below:

• Refine Telemanipulation via Phantom Omni
• User friendly Telemanipulation user interface
• Telestration
• Bimanual and Bilateral Telemanipulation with two Phantom Omnis
• Bimanual and Bilateral Telemanipulation with da Vinci Master Console
• Virtual fixtures via Telestration
• Documentation / User manual / Detailed report

• Minimum: (Expected by April 2nd)
  1. Improve telemanipulation (Completed)
  2. Telestration using Omni (Completed)
  3. Documentation / User Manual (Completed)

• Expected: (Expected by May 9th)
  1. Tele-work station for Tele-manipulation/stration(Completed)
  2. Bilateral teleoperation (Framework is completed, in progress)
  3. Bimanual teleoperation with two Omnis and two Eye Robots (Framework is completed, waiting for second Eye-Robot)

• Maximum: (Completion date Pending)
  1. Virtual fixture definition via Telestration (Not Completed)
  2. Telemanipulation and Telestration via da Vinci Master Console (Not Completed)

Technical Approach: Telemanipulation

• Modify Telecomponent App
• Based on cisstMultiTask Library
• Interfaces to existing robotic components including Omni and Eye Robot
• Runs tele-operation algorithms and logic (e.g. clutching)

Here is a diagram indicating the way that the teleopcomponent app connects to other system components.

The basis of telemanipulation lies in a “mimic” algorithm. The position of the Omni at Time1 and Time2 are considered, the delta is computed, and the Eye Robot is then moved with those delta values:

Technical Approach: Telestration

The steps for telestration are indicated in the following diagram. Note the following steps:

1. Obtain positions in the Omni coordinate system
2. Scale the positions by a scaling factor and consider the location of the Omni to be the point on the 3D display such that the calibrated Omni position is the center of the 3D display
3. Use the inverse of the existing framework that computes the transformation from Robotic coordinate system to 3D display in order to convert 3D display values to robotic coordinate system.

Once we have the points of the Omni in the Robotic coordinate system, we can develop virtual fixtures as follows. We can create a 3 dimensional space defined by the Omni in the Robotic coordinate system. Since we know the position of the tool from the Eye Tracker, we can then determine if, at any point, the tool of the Robot is within the allowed virtual fixture. The diagram below indicates this process:

• Eye Robot 2
  • Almost always available at nights and weekends
  • Med Campus Eye Robot 2
  • Can expect to use Eye Robot 1

• Omni
  • Safe to assume there is always one available

• 3D Video Display
  • Older technology available all the time (2D display)
  • Will use newer technology when available
  • Can use 2D for debugging

• Marcin
  • In the lab 7 days a week

• Access to the Lab
  • Resolved (have access)

• Access to Med Campus Lab
  • Not Resolved (no need)

• EYE-BRP SVN
  • Resolved (have access)

• da Vinci Master Console
  • Will resolve as needed

1. Milestone name: Fix the telemanipulation error
   • Expected Date: 3/5/12
   • Status: Completed on time
2. Milestone name: Telestration
   • Expected Date: 4/2/12
   • Status: Completed on time
3. Milestone name: Bilateral telemanipulation
   • Expected Date: 5/9/12
   • Status: Framework is completed, in-progress
4. Milestone name: Tele-work station
   • Expected Date: 4/30/12
   • Status: Completed a week late
5. Milestone name: Bimanual Teleoperation
   • Expected Date: 5/10/12
   • Status: Framework is completed, in-progress
6. Milestone Name: Documentation
   • Expected Date: 5/10/12
   • Status: Completed on time

• Project Plan
  • Project Plan Presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
• Robert Eisinger's Seminar Paper
  • Paper (Kumar et al.)
  • Critical Review
  • Paper Presentation
• Orhan Ozguner's Paper Presentation
  • Paper (Funda et al.)
  • Critical Review
  • Paper Presentation

• Project Checkpoint
• Project checkpoint presentation
• Project Final Presentation
• Final Poster
• Project Final Report
• Final Paper
• Example Videos
• Movie 1

• Uneri et. al., “New Steady-Hand Eye Robot with Micro-Force Sensing for Vitreoretinal Surgery,” IEEE RAS & EMBS, 2010.
• Balicki et. al., “Prototyping a Hybrid Cooperative and Telerobotic Surgical System for Retinal Microsurgery,” 2011.
• Ammi et. al., “Robotic Assisted Micromanipulation System using Virtual Fixtures and Metaphors,” IEEE Int. Conference, 2007.
• Kazanzides et., al., “Component-based software for dynamic configuration and control of computer assisted intervention systems,” 2011.
• Bohn et. al., “User interface integration and remote control for modular surgical assist systems,” 2010.
• J. Funda, R. Taylor, B. Eldridge, S. Gomory, and K. Grube, “Constrained Cartesian motion control for tele-operated surgical robots,” IEEE Transactions on Robotics and Automation, vol. 12, pp. 453-466, 1996.
• Additional readings on Ali Uneri and Gorkem Sevinc CIS 2 final project report on “Tele-operation of the Eye Robot”
• Additional readings on Seth Billings and Ehsan Basafa CIS 2 final project report on “Tele-operation of LARS Robot”

Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space.