Last updated: 5th March 2017

In this project, we will work on the platform IRIS2.0, a four motor actuated small-scale snake manipulator designed for eye surgery. Previous work have enabled the snake to move in both pitch and roll directions, in such way that only one of the four motors actuates the snake, instead of all four motors acting simultaneously. Our work aims to realize four-motor control as well as interface IRIS2.0 with haptic device Phantom Omni.

• Students: Baichuan Jiang, Yichuan Tang
• Mentor(s): Ehsan Azimi, Peter Kazanzides, Iulian Iordachita

Retinal microsurgery is one of the most technically challenging and high consequence surgical disciplines, which includes procedures that treat retinal diseases such as retina detachment, macular degeneration, diabetic retinopathy, epiretinal membrane, and retinal vein occlusion. In these procedures, surgeons need to cope with challenges raised by confined intraocular space and physical constraints in tool manipulation. Besides, very fine and precise control of the instruments is crucial to avoid complications on the delicate retinal tissues. Hand tremor may result in accidental damage to eye tissues or the undesired movement of eye ball. Therefore, based on the techniques of remote center-of- motion (RCM) and cooperative control, a Steady Hand Eye Robot (SHER) was developed to reduce hand tremor and provide RCM point at the sclera [1]. However, RCM will impose a strict limit on the tool orientation and the angle of approach between the surgical tip and the retinal tissue. Therefore, to enhance the manipulability of the intraocular tool, a more flexible design is demanded.

IRIS snake manipulator is developed by JHU based on a design proposed by Yong-Jae Kim and etc. ,a work supported in part by Samsung Electronics. The original design is named as “A stiffness-Adjustable neutral-line Manipulator”. Unlike traditional snake manipulator, it doesn’t have a flexible backbone or pivot or spring to connect each section of the mechanism, thus brought reduction to manufacture complexity. Such a snake robot could be controlled by four cords to achieve both pitch and roll rotations. These two features provide the advantage that IRIS could be built in a very small scale, and be applied into surgery which requires minimal invasion. The first prototype of IRIS manipulator was developed by Dr. Iordachita’s previous PhD student, Xingchi He. He defined the basic mechanical design and actuation system. Two years later, an upgraded version of IRIS: IRIS 2.0 was developed by Dr. Jingzhou Song and other researchers. That is the platform we will be working on. The new version uses better actuation system and has a smaller size, which allows its integration with SHER.

1 Our team will finalize the IRIS actuation unit and write a basic GUI, which could be used to power four motor at the same time to pull and release cords.

2 Experiment will be conducted to obtain a discrete mapping between bending angles of IRIS (pitch & roll) and the amount of motor rotations, over the entire workspace.

3 From data obtained in the experiment, we will establish an approximated continuous mapping between IRIS bending angles and motor angles. Then integrate this mapping with GUI so that user can input pitch and roll angles directly to control IRIS.

4 Interface Phantom Omni, so that when control units receive signal from this device, IRIS will move according to user’s intuitive manipulation.

5 Conduct further experiment and analyze experimental data to assess efficiency and accuracy of Phantom Omni interfaced IRIS.

The four-motor control enables IRIS to move to desired pose with only one command, other wise we have to input as least two commands (one for pitch, one for roll), and IRIS will go through at least two movements to get desired position, which is not usable in real surgical practice. The mapping between IRIS poses and motor rotation angles is key of the control algorithm, without it four motor simultaneous control is impossible. The interface of IRIS and Phantom Omni allows users to control IRIS in an intuitive way without the need to input IRIS pose information into the computer.

• Minimum: (Expected by 3/28)
  1. IRIS actuation GUI in which users can input four motor rotation angles and motors will rotate corresponding angles simultaneously.
  2. A discrete mapping between IRIS pitch & roll angles and motor rotation angles obtained by experiment.
  3. An approximated continuous mapping between IRIS pitch & roll angles and motor rotation angles, obtained by further processing of discrete mapping.

• Expected: (Expected by 4/16)
  1. Codes of the upgraded IRIS actuation GUI in which users can input IRIS pitch & roll angles and motor will rotate corresponding angles.
  2. Codes which interfaces Phantom Omni with IRIS.
  3. Video demonstration: manipulation of IRIS with Phantom Omni.

• Maximum: (Expected by 4/30)
  1. Analysis report of efficiency/accuracy evaluation data.
  2. Qualitative user feedback from ophthalmology surgeons.

Based on the data sheets of electronic control components, our team will use c++ programming environment to test the motor control api (and we will possibly refer to the previous control codes to get started). Our team will then develop a first version of IRIS actuation GUI using C++ on Qt platform. In this GUI user can input motor rotation angles to command 4 motors to rotate simultaneously. Next we will do experiment to test GUI by input a set of motor angles and measure angles rotated by the motor, then compare inputs and measurements, also conduct error analysis.

By actuating the 2-motor pairs in discrete steps, we can record and plot the mapping from the experiment. This is a preparation step for further experiment to find the full-workspace mapping between IRIS bending angles and 4-motor rotation angles (which represent wire translation distance). In the experiment, our team will input a set of motor angles into GUI, use cartesian stage, angular coordinate board and camera to record pitch and roll angles of IRIS. The size of data records is around hundreds.

We can use the same methodology as above. After obtain a discrete mapping between IRIS bending angles and motor angles in the full workspace of IRIS, the team will proceed to make a continuous version of such mapping, possibly through surface-fitting optimization techniques. It will be realized either by the available functionalities of MATLAB or by algorithms developed by the team. The purpose is to allow flexible input of IRIS pitch and roll angles, and as long as the input is within working range, motors will respond reasonably. The mapping is a one-to- one mapping, thus for a certain set of pitch and roll angles, there’s only one combination of motor angles. Then GUI code will be modified so that user can also input pitch and roll angles of IRIS besides angles of four motors. Then the new functionality will be tested with cartesian stage, angular coordinate board and camera. For each experiment, both pitch and roll angles need to be recorded. After reviewing experiment results, mapping algorithm will be modified to achieve better accuracy.

First an original point will be defined in Phantom Omni. The team will define a mapping between Phantom Omni movement and IRIS bending angles. Codes will be written so that GUI receive input from Phantom Omni to drive motors. The GUI will then need to be modified to have two working modes, one is that users could manually input bending angles, the other is that users can manipulate IRIS through Phantom Omni. Then, experiment will be carried out to test the accuracy and efficiency of the Phantom Omni interfaced IRIS, as well as tuning the mapping between Phantom Omni movement and IRIS bending angles. Finally, some Ophthalmology surgeons will be invited to use the integrated system and provide their qualitative feedback.

- Access to Phantom Omni & codes library

- Access to IRIS together with its motor controllers

- Access to controller documentation and C++ codes

- Access to a lab computer in LCSR

- Devices for conducting experiments

- availability of surgeons or other users

1. Milestone name: Control of individual motor through the motorAPI. Setup of programming environment.
   • Planned Date: 2/17/2017
   • Expected Date: 2/17/2017
   • Status: accomplished
2. Milestone name: GUI setup for the Row/Pitch control and direct motor control.
   • Planned Date: 2/28/2017
   • Expected Date: 3/7/2017
   • Status: on going
3. Milestone name: Calibration of Row/Pitch control under 2-motor driving condition
   • Planned Date: 3/7/2017
   • Expected Date: 3/7/2017
   • Status: not started
4. Milestone name: Design and setup of the 4-motor simultaneous control law with calibration.
   • Planned Date: 3/28/2017
   • Expected Date: 3/28/2017
   • Status: not started
5. Milestone name: GUI integration with Phantom Omni as input source
   • Planned Date: 3/16/2017
   • Expected Date: 3/16/2017
   • Status: on going
6. Milestone name: Analysis of experiments for validating the feasibility, accuracy, repeatability, efficiency.
   • Planned Date: 4/30/2017
   • Expected Date: 4/30/2017
   • Status: not started

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentations
  • here provide links to all seminar presentations
  • cis_ii_seminar_report_-_baichuan.pdf
  • cis_ii_seminar_report_yichuan.pdf
• Project Final Presentation
  • PDF of Poster
• Project Final Report
  • Final Report
  • links to any appendices or other material

1. Üneri, Ali, et al. "New steady-hand eye robot with micro-force sensing for vitreoretinal surgery." Biomedical Robotics and Biomechatronics (BioRob), 2010 3rd IEEE RAS and EMBS International Conference on. IEEE, 2010.

2. He, X., van Geirt, V., Gehlbach, P., Taylor, R., & Iordachita, I. (2015, May). Iris: Integrated robotic intraocular snake. In Robotics and Automation (ICRA), 2015 IEEE International Conference on (pp. 1764-1769). IEEE.

3. Song, J., Gonenc, B., Guo, J., & Iordachita, I. (2017, May) Intraocular Snake Integrated with the Steady-Hand Eye Robot for Assisted Retinal Microsurgery. In Robotics and Automation (ICRA), 2017 IEEE International Conference on. IEEE.

4. Y.-J. Kim, S. Cheng, S. Kim, and K. Iagnemma, “A stiffness-adjustable hyperredundant manipulator using a variable neutral-line mechanism for minimally invasive surgery,” IEEE Transactions on Robotics, vol. 30, no. 2, pp. 382–395, Apr. 2014.

5. "Control of the coupled motion of a 6 DoF robotic arm and a continuum manipulator for the treatment of pelvis osteolysis," in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2014, pp. 6521-6525.

6. N. Simaan, “Snake-like units using flexible backbones and actuation redundancy for enhanced miniaturization,” in IEEE International Conference on Robotics and Automation, no. April, 2005, pp. 3012–3017.

7. P. Gupta, P. Jensen, and E. de Juan, “Surgical forces and tactile perception during retinal microsurgery,” in Proc. MICCAI’99, 1999, pp. 1218–1225.

8. S.E. Boye, S.L. Boye, A.S. Lewin, and W.W. Hauswirth, “A comprehensive review of retinal gene therapy,” Mol. Ther.21(3), pp. 509-519, 2013.

9. J.N. Weiss, and L.A. Bynoe, “Injection of tissue plasminogen activator into a branch retinal vein in eyes with central vein occlusion,” Ophthalmology, vol. 108(12), pp. 2249-2257, July 2001.

10. B. Gonenc, P. Gehlbach, J. Handa, R.H. Taylor, and I. Iordachita, “Force-Sensing Microneedle for Assisted Retinal Vein Cannulation,” in Proc. IEEE SENSORS 2014, 2014, pp. 698-701.

11. E. Vander Poorten, “Design and realisation of a novel robotic manipulator for retinal surgery,” in Proc. IEEE Int. Conf. on Intelligent Robots and Systems, Tokyo, 2013, pp. 3598-3603.

12. B.C. Becker, S. Yang, R.A. MacLachlan, and C.N. Riviere, “Towards vision-based control of a handheld micromanipulator for retinal cannulation in an eyeball phantom,” in Proc. 4th IEEE RAS EMBS Int. Conf. Biomed. Robot. Biomechatron., 2012, pp. 44-49.

Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space.