Enter a short narrative description here

• Students: Muhammad Hadi, Mojtaba Esfendiari, Trent Tang*, Abudllah Al Armouti*
• Mentor(s): Dr. Iulian Iordachita, Ali Ebrahimi, Dr. Alireza Alamdar, Dr. Adnan Munawar
  • *- Non CIS II affiliates

Our project revolves around implementing a teleoperation algorithm to teleoperate a 7 Degree-of-Freedom vitreoretinal surgical platform with a 5 Degree-of-Freedom Phantom Omni.

The goal is to successfully identify an optimal algorithm that allows for smooth and intuitive control of the 7 DOF hybrid system, implement said algorithm, and test it on a custom experimental setup, as a precursor to potential clinical applicaitons.

Envisioned high dexterity intraocular manipulator: (A) Epiretinal membrane peeling; (B) Steady Hand Eye Robot; (C) Integrated robotic intraocular snake robot; (D) Phantom Omni; (E) Distal snake-like tool-end inside eye phantom.

Retinal surgery is a highly delicate and difficult surgery, revolving around surgery performed deep within the eye, on the surface of the retina. Examples of this surgery are epiretinal membrane peeling and retinal vein cannulation, to name a few.

However, there are several factors that could be a potential risk during such surgeries. The primary issue arises from physiological hand tremor, which, as faint as it may be, can still result complications during surgery. For example, when peeling back scar tissue during an epiretinal membrane peeling, any slight jerks form hand tremor can result in retinal tears. Similarly, exerting forces > 7.5 mN to the retina could result in retinal tears, forces which are too fine to be felt by a human. All these factors result in an exceptional level of training that surgeons have to undergo to be able to perform these surgeries.

Therefore, to resolve the above stated issues, researcher at Johns Hopkins University, AMIRO LCSR, came up with a surgical robotic system, a Steady Hand Eye Robot (SHER), which performs robot-controlled surgery, allowing surgeons to mitigate (filter) physiological tremor affects and provide them with haptic force feedback.

However, even with SHER, there are still some operative procedures that may not be easily executable, given the limited flexibility of the end effector. A straight needle attached to the end-effector of SHER does not provide enough dexterity for surgeons operating inside the eye, with surgeons often having to move the eye to navigate. Therefore, a 2 DOF Integrated Robotic Intraocular Snake (IRIS) was designed and attached to the SHER end-effector. This eliminated the need to move the eye by employing a more flexible end-effector.

Minimum

• Evaluating the already existing software of the snake robot and eye robot + tele-operation control of both
• A short report detailing successful integration of the Snake Robot software with the Eye Robot software
• A mathematical model of accurate mapping of snake robot movement/calibrate I2RIS

Expected

• Code package with successful control algorithms for the tele-operated system with constraints
• A report highlighting the results of experimental testing of the implemented teleoperation code

Maximum

• Design and execution of experiment to evaluate complete teleoperation control vs cooperative control
• Academic paper on teleoperation control of hybrid system (Eye + Snake Robot)

To design a control algorithm for the snake robot, we need to have knowledge about the robot kinematics. Given that kinematics modeling of continuum robots is not as straightforward as conventional manipulators, we aim to develop an experimental forward kinematics and calibrate the movement of the new snake robot (I2RIS) to generate a mapping between the snake robot actuation space and its configuration space and finally to its task space. This will allow for a precise forward kinematic mapping, which is critical to establish any form of inverse kinematic model.

Developing a mapping between the linear displacement of the cables (actuation space),the bending angle of the IRIS robot (configuration space) and the IRIS robot tip location (task space

To develop a teleoperation control algorithm, we need to integrate the snake code package (CISST-SAW) and that of the eye robot via ROS. Then, using a 5 DOF Phantom Omni as a local robot, we want to control the hybrid 7 DOF Snake & Eye Robot (I2RIS + SHER 2.1) as remote robot. To do this, we intend to design an optimal inverse kinematics method with new constraints of the hybrid system. There are several constraints that should be considered in the teleoperation algorithm such as robot joint limits, motor velocity limits, and RCM constraint.

​ .

By utilizing the setup above, we moved our snake with specific encoder values, collected image data, and used the Color Thresholder MATLAB toolbox to pass our collected images through a round of image analysis to track the position and angle of the optical fibre that was inserted through the snake.

As can be observed, the results are not satisfactory, as can be inferred from the extensive hysteresis, and and unusual horizontal behavior near the neutral axis. We plan to repeat this calibration with a better built snake

Accomplished teleoperation control of Eye-Robot in RCM mode, which is a condition that has to be met when the system has entered the eye. In conjunction with the teleoperation of the SNAKE robot, this allows for compartmentalized Teleoperation control.

— mhadi2 2022/03/31 14:48

1. Milestone name: Integrating code packages
   • Planned Date: 03/14
   • Expected Date: 03/14
   • Status: ONGOING
2. Milestone name: Calibrate Snake model
   • Planned Date: 03/07
   • Expected Date: 03/07
   • Status: COMPLETED
3. Milestone name: Kinematic model of 7 DOF system
   • Planned Date: 03/21
   • Expected Date: 03/21
   • Status: ONGOING
4. Milestone name: Implementing Teleoperation Algorithm
   • Planned Date: 03/28
   • Expected Date: 03/28
   • Status: ONGOING
5. Milestone name: Testing Teleoperation Algorithm
   • Planned Date: 04/11
   • Expected Date: 04/11
   • Status: Possibly delayed till summer

— mhadi2 2022/03/31 14:59

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • Project Background reading.
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentations
  • Critical Review
• Project Final Presentation
  • PDF of Poster
• Project Final Report
  • Final Report
  • links to any appendices or other material
• Snake Calibration Report
  • Snake Calibration Report
• Teleoperation Workflow
  • Teleoperation Workflow Instructions

• GITHUB page
  • CIS-II-GITHUB

He, X., Van Geirt, V., Gehlbach, P., Taylor, R., & Iordachita, I. (2015, May). IRIS: integrated robotic intraocular snake. In 2015 IEEE International Conference on Robotics and Automation (ICRA) (pp. 1764-1769). IEEE​

Song, J., Gonenc, B., Guo, J., & Iordachita, I. (2017, May). Intraocular snake integrated with the steady-hand eye robot for assisted retinal microsurgery. In 2017 IEEE International Conference on Robotics and Automation (ICRA) (pp. 6724-6729). IEEE.​

Üneri, A., Balicki, M. A., Handa, J., Gehlbach, P., Taylor, R. H., & Iordachita, I. (2010, September). New steady-hand eye robot with micro-force sensing for vitreoretinal surgery. In 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (pp. 814-819). IEEE.​

He, C., Ebrahimi, A., Roizenblatt, M., Patel, N., Yang, Y., Gehlbach, P. L., & Iordachita, I. (2018, August). User behavior evaluation in robot-assisted retinal surgery. In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (pp. 174-179). IEEE.​

Ebrahimi, A., He, C., Roizenblatt, M., Patel, N., Sefati, S., Gehlbach, P., & Iordachita, I. (2018, July). Real-time sclera force feedback for enabling safe robot-assisted vitreoretinal surgery. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 3650-3655). IEEE.​

Wilkening, P., Alambeigi, F., Murphy, R. J., Taylor, R. H., & Armand, M. (2017). Development and experimental evaluation of concurrent control of a robotic arm and continuum manipulator for osteolytic lesion treatment. IEEE robotics and automation letters, 2(3), 1625-1631.

Penning, R.S., Jung, J., Ferrier, N.J. and Zinn, M.R., 2012, May. An evaluation of closed-loop control options for continuum manipulators. In 2012 IEEE International Conference on Robotics and Automation (pp. 5392-5397). IEEE.​

He, C., Ebrahimi, A., Roizenblatt, M., Patel, N., Yang, Y., Gehlbach, P. L., & Iordachita, I. (2018, August). User behavior evaluation in robot-assisted retinal surgery. In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (pp. 174-179). IEEE.​

Dimeas, F., & Aspragathos, N. (2016). Online stability in human-robot cooperation with admittance control. IEEE transactions on haptics, 9(2), 267-278.​

Shi, Kaiyu; Zhou, Yishun; Ebrahimi, Ali; Li, Gang; Iordachita, Iulian (2022), Optimization-based Concurrent Control of a High Dexterity Robot for Vitreoretinal Surgery. Manuscript submitted for publication​

Rahimy, E., Wilson, J., Tsao, T. C., Schwartz, S., & Hubschman, J. P. (2013). Robot-assisted intraocular surgery: development of the IRISS and feasibility studies in an animal model. Eye, 27(8), 972-978.​

P. Gupta, P. Jensen, and E. de Juan, “Surgical forces and tactile perception during retinal microsurgery,” in International Conference on Medical Image Computing and Computer Assisted Intervention, vol. 1679, 1999, pp. 1218–1225​

Ebrahimi, Ali. (2018). Robot Control Algorithms Based on Sclera Force Information [PowerPoint presentation], CIS II. ​

Ebrahimi, Ali. (2018). CIS II Project – Spring 2022 [PowerPoint presentation], CIS II. ​

Song, J., Gonenc, B., Guo, J., & Iordachita, I. (2017, May). Intraocular snake integrated with the steady-hand eye robot for assisted retinal microsurgery. In 2017 IEEE International Conference on Robotics and Automation (ICRA) (pp. 6724-6729). IEEE.