Contact Us
CiiS Lab
Johns Hopkins University
112 Hackerman Hall
3400 N. Charles Street
Baltimore, MD 21218
Directions
Lab Director
Russell Taylor
127 Hackerman Hall
rht@jhu.edu
Last updated: 5/10/23 10:30 am
This proposal presents work at the intersection of continuum robot development and retinal surgery to create a device for safe subretinal injection by developing a robot and controller for a novel tendon-only manipulator, the continuum wire manipulator (CWM). The proposed technology utilizes flexible continuum structures and precise control of state-of-the-art continuum methodologies. The research in this work will need to meet navigation, size, and force constraints of our novel subretinal surgery approach which includes navigation between scleral and choroidal layers to arrive at the subretinal space. The device must also avoid puncturing the key visual structure inappropriately and make both simple S-curve and C-curve shapes.
The retina is a “layer of nervous tissue that covers the inside of the back two-thirds of the eyeball, in which stimulation by light occurs, initiating the sensation of vision” and “is actually an extension of the brain, formed embryonically from neural tissue and connected to the brain proper by the optic nerve”. Any damage to the retina may cause irreversible and permanent visual field defect or even blindness.
Retinal surgery has long drawn the attention of engineers and clinicians who identified a clear use case for robotics and assistive technology. This technology is challenging to make, however, because of the requirement to be very extremely small, delicate, and precise. In retinal surgery, skilled practitioners operate on the boundaries of human capability, dealing with minuscule anatomic structures that are both fragile and hard to discern. Surgical operations on the retina, a hair-thick multilayered structure that is an integral part of the central nervous system responsible for vision, spurred the development of robotic systems that enhance perception, precision, and dexterity.
Continuum robots are more recent developments in the robotics world with non-rigid bodies shown to be able to be designed in very miniscule manners. Continuum robots have been shown to be particularly effective at navigating through tortuous anatomical pathways in the human body. Some have shown promise of steering along 3D curves in confined spaces and dexterously handle tissues. The two most popular and used continuum robots in the medical world are tendon-driven robots and continuum tube robots (CTCRs). CTCRs comprise a series of pre-curved elastic tubes that can be translated and rotated with respect to each other to control the tube manipulator shape and tip pose. It is a rapidly maturing technology that has seen extensive research over the past decade . Today, they are being evaluated as tools for a variety of surgical applications, as they can offer precision and manipulability in tight workspaces.
Over 200 million people worldwide suffer from some form of retinal degenerative diseases (RDDs) (primarily the aging population over 55). These RDDs include wet and dry age-related macular degeneration (AMD) and Stargardt’s disease among others, and cause debilitating blindness. Current pharmaceutical drugs made by large pharmaceutical companies claim their drugs can heal these disease, however Phase I/II trials have all been unsuccessful causing serious adverse effects (SAEs). These trials and research today use trans-retinal approaches to access the retina. To inject biologics to the subretinal space, this access method requires a hole to be made through the retina so drugs can be injected. Unlike the sclera, piercing through the retina is traumatic and does not readily heal.
Alternatively, trans-scleral approaches into the suprachoroidal space have been proposed and validated previously to access the subretinal space. Access via this route would avoid causing SAEs which come from trans-vitreal methods. This method, however, requires very thin, delicate tools, which can be steerable and easily deformable as opposed to the motion of traditional trans-vitreal needles commonly used for surgery, which are rigid. Here we plan to use a flexible continuum robot to safely access the subretinal space so a future minimally-invasive injection can be performed. This requires that the tool avoid puncturing key visual structures. Successful completion of this project will allow for safer injection for robot-assisted subretinal surgery. It also has the potential to carry tools attached to the end effector (such as cannulas for drug delivery, or cameras) for a variety of other surgeries and surgical tasks.
The design of a surgical robot is a complex task that involves the integration of different components and technologies. In particular, mechanical design plays a crucial role in ensuring the accuracy and precision of the robot's movements.
One of the key components of this project is the continuum end effector, which provides the necessary dexterity and flexibility to perform complex surgical tasks. The continuum end effector is designed to be easy to curve and slide, which allows it to navigate through narrow and complex anatomical structures. It has four degrees of freedom, which are controlled by servo motors.
To achieve precise and reliable control of the rotation motors, we are using Maxon DCX 8 M motors, which are known for their high performance and reliability. These motors are equipped with Maxon GPX 8 gearboxes and Maxon ENX 8 MAG encoders, which provide accurate and precise feedback on the motor's position and speed. The linear motor Maxon RE 8 is used to provide linear motion to the robot's end effector. This motor is designed to be compact and lightweight, while still providing high performance and accuracy.
To control the motion of the motors, we are using Maxon EPOS2 24/2 motion controller, which provides advanced features such as basic trajectory planning, position and velocity control, and real-time feedback. The motion controller will be a key component of the robot's control system, and it allows us to achieve precise and reliable motion control. Through careful design and integration, we will be able to achieve the necessary accuracy and precision to perform complex surgical tasks with the robot.
The robotic platform can be divided into three parts, the container for the rotation motors, the linear base for the translation motors, and the housing to fix the robot.
The overall length, width, and height of the container are 30, 11, and 34.5 mm respectively. The primary objective of the container is to accommodate the M8 rotation motors by means of an upper hole with a radius of 4.25 mm. A hexagonal nut of M8 size is fixed in the front with the help of a soldering iron to secure the motor. A small wire holder is introduced to link the end of the rotation motor with the continuum wire. The lower part of the container is designed with a hole of outer and inner radii of 3.5 mm and 2.65 mm, respectively, to accommodate the leading screw and hence transmit linear motion. The connecting section of the screw is M5.5, and accordingly, a thread tap of the same size is employed in the inner ring to retain the screw. Additionally, a small hole with a radius of 1.35 mm is utilized to hold set screws that keep the leading screw stable. On the base, there exist two holes with a radius of 1.35 mm and a spacing of 6.5 mm. These holes are intended to hold M2 thread inserts and align with the slide units of the IKO LWLF standard linear path. There is a hole of the same size going through the container from the upper side to the base, which gives access to help installing the screw in the M2 inserts. A triangular structure is applied on both sides of the base to enhance stability.
The second significant part is the linear base, of which the length, width, and height are 78, 11, and 30 mm respectively. The upper hole is designed with radius 4.25 mm and installed with a hexagonal M8 nut to hold the linear motor. The left and right boundaries are 0.5 mm to keep two wires (from the neighbor motor) as close as possible. Hence, a triangular structure is designed to enhance stability of the thin motor container. Two holes of diameter 5.9 mm and spacing 20 mm are designed in the bottom to contain the M4 thread inserts, which would help fix the base in the housing structure. In the front is a horizontal platform of length 50 mm to get IKO LWLF standard linear path installed. The IKO linear path can avoid torque in the pitch direction and has rolling balls inside the slide units, which could help smoothen the motion and keep the horizontal stability. Four holes of radius 1 mm and spacing 10 mm are set to accommodate the M1.6 thread inserts and help fix the linear path. In the very front, a baffle of height 20 mm is designed to keep the linear motion in the desired range, which is 39.1 mm determined by the length of the IKO linear path.
The third part is the housing structure, which is a grooved cuboid with holes for the M4 screws on both sides. Currently, we use it to fix two single robots together and keep the wire spacing as 11.5 mm. A nano motor interface is designed to integrate with Steady Hand Eye Robot (SHER) and expected to mount on the housing structure.
The parts mentioned are generated by 3D printer (Stratasys F170, polylactic acid, 60% fill), SLA printer (Formlabs, stereolithography), and CNC manufacture. All the detailed parameters and the logic to set up such parameters can be found in the CAD_readme file on the project wiki page.
Motor and motion control libraries are widely used in robotic applications and provide a straightforward way to directly interface with the hardware. Before calling the library, we should first make the mechanical part wired with the controller. Here we use two types of settings for rotation and translation motors.
The rotation is generated by Maxon DCX 8M (precious metal brushes, DC motor) with gear box Maxon GPX 8 (planetary gearhead) and encoder Maxon ENX 8 MAG. It needs a power supply of 4.2 and 3.3 V to actuate the motor and the encoder respectively. The maximum speed (no-load speed) and the maximum continuous current (Nominal current) can reach 11700 rpm and 199 mA respectively. The channel number of the incremental encoder is 3, which is A, B, and I. A and B are the two primary output channels providing quadrature signals to determine the direction of rotation and the relative position of the encoder. Channel I generates a pulse once per revolution and is typically used to reset the position count to a known value or to indicate a specific position or event. It is a relatively encoder with 256 counts per turn.
The translation is generated by Maxon RE8 (precious metal brushes, DC motor) with screw drive Maxon GP 8 S (metric lead screw) and encoder Maxon MR. It needs a power supply of 6 and 5 V to actuate the motor and the encoder respectively. The maximum speed (no-load speed) and the maximum continuous current (Nominal current) can reach 13300 rpm and 155 mA respectively. The channel number of the incremental encoder is 2, which is A and B. It is a relatively encoder with 100 counts per turn.
Our goal is to control two wires simultaneously. Hence, we should get in communication with four motors. The Maxon controller EPOS2 24/2 (390438) is applied to play a role as human machine interface. It is power by Mastech DC Power Supply HY3005F-3 in a range of +9 to +24 V and is capable of automatically adjusting the motor power demands. We use the jumper J9 which has 10 pins corresponding to motor+, fixed 5V/100mA sensor supply, ground, motor-, channel A bar, channel A, channel B bar, channel B, channel I bar, and channel I respectively. The controller interacts with the computer by USB interface. We use SABRENT 4-Port USB Hub to facilitate real-time monitoring of 4 separate motor conditions.
All the wiring to a single is integrated in a encoder wire Maxon ENX MAG. The rotation motor requires 12-pin inputs, while the translation motor requires 8-pin inputs. However, the output order of the 10-pin controller does not match either of these requirements. To address this issue, we utilized the Maxon Micromotor Adaptor, which enables 12-pin reorganization of the connections via jumper (J3 with M+ M- closed and J4 with 3.3 V closed). The last two channels of J3 are shielded in order to stay in touch with the translation motor. Further details regarding the wiring plan can be found on the wiki page.
All other detailed parameters can be found on the wiki page.
In order to achieve precise motion of a robot, it is necessary to develop both low-level and high-level control . This project primarily focuses on developing low level control integration for basic motions controlled by a few high-level code snippets. The low-level control is responsible for directly interacting with the actuation of the robot, including individual motor control. We use integrated motor control capabilities for PID (Proportional-Integral-Derivative) control for tuning of individual motors in preparation for future control schemes to be implemented for fully integrated robot control (most likely model predictive control or similar).
The four separate controllers are connected to the computer via a USB hub, and each channel is configured to achieve individual control. The parameter known as quadrature count (QC), which represents the number of pulse counts generated by the quadrature output signals of an encoder for a given rotation of the encoder shaft, is automatically obtained by the software and helps in accurately tracking the position and velocity of rotating machinery. The built-in control parameters are then tuned by evaluating their step response. Once all the setups are complete, the operator can precisely control the current, velocity, and position of the motor.
To implement the low-level control, the EPOS control libraries will be leveraged, as they provide a set of preset software tools for basic motion. Via EPOS studio, the operator is able to custom control orders, including homing, (profile / interpolated) position, (profile) velocity, current, and step direction mode etc.. Since the counterclockwise rotation is the positive direction, the translation motor steps backward with a positive velocity. In our future work, we aim to assess the kinematic model by adapting the position and velocity mode, and the dynamic model by implementing the current mode. Deep learning and finite element analysis may be potential methods to estimate models. The primary function is the current control, which forms the basis for the secondary position and velocity control. The advanced functions are subsequently developed upon this foundation. The detailed commands can be found in the attached control library.
Following the mechanical design, wiring plan, and the control configuration, we successfully build up the system and ready to showcase the robot motion. By estimating the forward kinematics, we develop three motion modes, twist, curl, and yaw motion. Applying the same velocity in the rotation shifts, the wire is able to move in the roll direction and twist a node which allows the wire slide and curve in a desired shape. After the node creation, apply the opposite velocity in the rotation shifts and the wire loop will move in the pitch direction, which is defined as the curl mode. Moving the node in the opposite linear direction will actuate the wire loop in the yaw direction.
In the testing section, we first make the linear motors align to each other and arrange the wire in a loop shape, which is defined as home position. Then both rotation motors are set to be 300 rpm in the counterclockwise direction to twist a node. The velocity profile is set to be a trapezoidal shape. The rotation motors (#1 and #2) are then set to be 300 and -300 rpm respectively to make the wire loop curl upward at the node. Conversely, the wire loop could curl downward at the node with -300 and 300 rpm. We control the translation motor #1 and #2 to move in 5000 and -5000 rpm respectively to make the wire loop yaw clockwise. The counterclockwise yaw is set vice versa.
Once the low-level control has been implemented and tested, the high-level control basic commands can be developed to integrate the robot with a whole surgery system. One popular library which contains some tools for high-level control is the ROS (Robot Operating System) control library. ROS is a widely used framework for building robotic systems and provides a set of tools and libraries for developing distributed robotic systems. The CISST (Computer-Intuitive Surgical Systems and Technology) system is another JHU-based library that can be used for developing high-level control of surgical robots. The CISST system is a modular and flexible system that allows the integration of various components and subsystems of a surgical robot. High level control for this system will be implemented in CISST and CRTK-inspired ways, though the full CISST System will not necessarily be implemented.
Our next experiment is to demonstrate the robot's motion in an agar gel environment, which simulates the conditions of the human body. It is essential to gather sensor feedback data during the motion to establish kinematic and dynamic models. Any unforeseen collisions, oscillations, or resistance encountered during the process will add modified terms to the model, which will aid in the final assessment of safe subretinal gene therapy delivery into a phantom pig eye. However, due to time constraints, this aspect will be addressed after the CIS lecture. In the future map, we would also use deep neural network and finite element analysis to calculate the mathematical model.
After studying the series work of Dr. Desai, which conducts research on tendon-driven COaxially Aligned STeerable robot (COAST) in peripheral vascular intervention task [9, 10, 11, 12]. We are inspired by the advancement mechanism, including spool, clamp, roller, and spur gear system etc., to control the wire length and shape simultaneously. The current linear path we are using is only 39.1mm long, which is not sufficient for wire feeding and yawing tasks. Therefore, we plan to explore the feeding and retracting strategies used in COAST to help meet our project requirements.
[1] Mitros, Z., Sadati, S. M. H., Henry, R., Da Cruz, L., & Bergeles, C. (2021). Annual review of control, robotics, and autonomous systems from theoretical work to clinical translation: Progress in concentric tube robots doi:10.1146/annurev-control-042920-
[2] Webster, R. J., & Jones, B. A. (2010). Design and kinematic modeling of constant curvature continuum robots: A review. London, England: SAGE Publications. doi:10.1177/0278364910368147
[3] Vander Poorten, E., Riviere, C. N., Abbott, J. J., Bergeles, C., Nasseri, M. A., Kang, J. U., … & Iordachita, I. (2020). Robotic retinal surgery. In Handbook of Robotic and Image-Guided Surgery (pp. 627-672). Elsevier.
[4] Fleming, I., Balicki, M., Koo, J., Iordachita, I., Mitchell, B., Handa, J., … & Taylor, R. (2008). Cooperative robot assistant for retinal microsurgery. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2008: 11th International Conference, New York, NY, USA, September 6-10, 2008, Proceedings, Part II 11 (pp. 543-550). Springer Berlin Heidelberg.
[5] Gijbels, A., Smits, J., Schoevaerdts, L., Willekens, K., Vander Poorten, E. B., Stalmans, P., & Reynaerts, D. (2018). In-human robot-assisted retinal vein cannulation, a world first. Annals of biomedical engineering, 46, 1676-1685.
[6] Üneri, A., Balicki, M. A., Handa, J., Gehlbach, P., Taylor, R. H., & Iordachita, I. (2010, September). New steady-hand eye robot with micro-force sensing for vitreoretinal surgery. In 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (pp. 814-819). IEEE.
[7] Ferrentino, P., Seyedreza, K. T., Brancart, J., Van Assche, G., Vanderborght, B., & Terryn, S. (2021). FEA-based inverse kinematic control: Hyperelastic material characterization of self-healing soft robots. IEEE Robotics & Automation Magazine, 29(3), 2-12. doi:10.1109/MRA.2021.3132803
[8] Kuntz, A., Sethi, A., Webster, R. J., & Alterovitz, R. (2020). Learning the complete shape of concentric tube robots. IEEE Transactions on Medical Robotics and Bionics, 2(2), 140-147. doi:10.1109/TMRB.2020.2974523
[9] Lis, P., Sarma, A., Trimpe, G., Brumfiel, T. A., Qi, R., & Desai, J. P. (2022, May). Design and Modeling of a Compact Advancement Mechanism for a Modified COAST Guidewire Robot. In 2022 International Conference on Robotics and Automation (ICRA) (pp. 1176-1182). IEEE.
[10] Chitalia, Y., Wang, X., & Desai, J. P. (2018, May). Design, modeling and control of a 2-dof robotic guidewire. In 2018 IEEE International Conference on Robotics and Automation (ICRA) (pp. 32-37). IEEE.
[11] Jeong, S., Chitalia, Y., & Desai, J. P. (2020). Design, modeling, and control of a coaxially aligned steerable (COAST) guidewire robot. IEEE Robotics and Automation Letters, 5(3), 4947-4954.
[12] Sarma, A., Brumfiel, T. A., Chitalia, Y., & Desai, J. P. (2022). Kinematic modeling and Jacobian-based control of the COAST guidewire robot. IEEE Transactions on Medical Robotics and Bionics, 4(4), 967-975.
Please contact Shuyuan Wang (swang340@jhu.edu) to get access to private links.
