Haptic Interface for Surgical Manipulator System

Last updated: 11 May 2012 at 5:36p

A surgical manipulator, the JHU/APL snake manipulator, intended to make hip osteolysis surgery more effective and faster with fewer complications has been developed. It has translational, rotational, and bend modes of motion, and its current controller utilizes a keyboard and mouse. On this page, we describe the successful development and implementation of two distinct interfaces for the manipulator using a PHANTOM Premium haptic controller, which aim to increase the intuitiveness of control. The first interface allows the user to select a target position and have the manipulator move to it. The second allows the user to continuously control the position of the manipulator. We also discuss our various additions and enhancements to the base interfaces, including the use of audio, visual, and force feedback.

<fs larger>Goal</fs>: Develop an intuitive haptic interface to control the end-effector position of an existing surgical manipulator.

• Students: Manish Mehta, Piyush Poddar, Jessie Young
• Mentors: Mehran Armand, Michael Kutzer, Ryan Murphy

The number of total hip arthroplasties (THAs) is expected to increase by over 174% in the next twenty years. Component wear and osteolysis, the active resorption of bone around components, are the primary phenomena responsible for the shortened life spans of total hip replacements. Wear of the polyethylene liner of the implant causes macrophage activation and subsequent bone resorption. This osteolysis comprises the integrity of the bone and thus introduces the need for frequent revision surgery to remove the lesion and maintain the previously-used fixed acetabular component. Osteolytic lesions of the bone around the implant, if not removed, may lead to complications such as bone fracture or component loosening or disconnection [3].

Minimally-invasive approaches aim at replacing the polyethylene liner while preserving the acetabular and femoral components of the THA. In this manner, surgeons minimize the risk of introducing bone fracture. Minimal-invasiveness is achieved by accessing lesions through screw holes in the bone drilled from the original implant. A major challenge, however, is fully accessing the entire volume inside the lesion to clean the cavity; studies have shown that on average less than half of the lesion is cleaned during manual procedures using curettes and other tools. Eventually, this lack of coverage forces the need for majorly invasive surgery in which the hip replacement is removed, the lesion is cleaned out, and another hip replacement is introduced [3]. Of course, an additional issue is that since the surgery site is being accessed through a screw hole during the minimally-invasive procedure, the surgeon also cannot visualize the lesion. He must use x-ray fluoroscopy, though this use must be limited to prevent overexposure of the patient and staff to radiation. Therefore, a highly dexterous manipulator that can cover the majority of the volume inside the lesion during the minimally invasive surgery as well as a way to track it is essential.

The Johns Hopkins University Whiting School of Engineering and Johns Hopkins Applied Physics Laboratory have developed such a cable-driven surgical manipulator system (see ‘Devices’ below). However, accurate and smooth operation of the manipulator is difficult, and the system lacks force feedback that allows the surgeon to “feel” his way around the cavity as he does during manual surgery. Since the system does not have a navigation system that allows the surgeon to orient himself within the lesion, he can only estimate where in the cavity the manipulator is currently located. Prior research in the surgical value of haptic feedback has shown that robotic systems incorporating haptic feedback can significantly improve surgical performance. Shortened operative times; increased consistency, precision, and performance; and decreased frequency of surgical errors are all commonly demonstrated benefits drawn from comparing robotic systems with and without haptic feedback [6][8]. With no consensus as to how best implement haptic feedback, a variety of haptic-deliverance paradigms have been implemented and evaluated. With the integration of haptics into the robotic control interface, we aim to bring the aforementioned benefits to the cable-driven surgical manipulator system during osteolysis revision surgery.

The primary objective of creating a haptic interface for the dexterous manipulator (DM) is to convert the user’s input into manipulator end behavior. Therefore, we seek to create a set of highly modular interfaces that 1) create a hierarchical system that abstracts away particular implementation details and 2) allow for more easily maintainable code. Our interface, therefore, will be controllable with different input devices (for example, the keyboard, another haptic device such as the PHANTOM® OMNI, or the SpaceNavigator® 3D Mouse), and can communicate with different motor controllers or even the manipulator itself. The ability to swap out different hardware and software components while still maintaining the same core functionality allows users to customize the system their needs. The DM, the snake-like portion of the robot, is controlled using the PMX-2ED-SA 2-axis stepper motor controller from Arcus Technology, Inc. The controller is interfaced with the PC via USB and contains two 10-bit analog inputs [3] that measure the force from each of the DM drive cables (recall that the pull of each of the cables will “wiggle” the tip of the snake either left or right while constrained within a single plane).

Two additional degrees of freedom are provided by a stage on which the DM is mounted. The stage is composed of a translational (Y—forward and back) actuator and a rotational (θ) actuator, both of which are controlled by DMX-UMD-23-3 integrated stepper motors from Arcus [3]. These also interface with the PC via USB. An application may control the motors by invoking an ASCII command set on the PMX and DMX C++ dlls.

The PHANTOM Premium is a haptic controller that allows for movements and force feedback in 6 DOF. Studies using this system have shown the benefit of using force feedback is two-fold: the applied forces can both act as a physical constraint on hand motion and as a guide to for planning intended motion [6][9]. The PHANTOM also reports positional and velocity measurements for the 6 DOF. Finally, the PHANTOM has a button on the stylus that can be toggled at the preference of the user.

Interacting with the PHANTOMis done through a C++ interface. Within this interface, multiple options are available. For example, force feedback can be applied to simulate a variety of different natural forces, including friction and viscosity. In addition, the PHANTOMcan be made to act as though the stylus tip is attached to a given point in space via a spring of given spring constant. Another mode of force feedback is used to constrain the workspace of the PHANTOM. A CAD model can be loaded and the PHANTOM will generate force feedback to simulate the dimensions and shape of the object. A visualization of the workspace is also generated.

The Johns Hopkins University Whiting School of Engineering and Johns Hopkins Applied Physics Laboratory have developed a cable-driven surgical manipulator system. Simulations of the system have suggested 85–95% coverage rates of surgically relevant osteolytic cavities [3], a significant improvement from traditional manual surgical methods.It has a snake-like cannula that can access osteolytic lesions through the lumen of a larger, rigid guide cannula [3].The actuation unit consists of a Y-θ stage with cable drive motors. The stage is composed of a translational (Y—forward and back) actuator and a rotational (θ) actuator, both of which are controlled by DMX-UMD-23-3 integrated stepper motors (ArcusTechnology, Inc.) [3]. The manipulator can also be bent in plane in either direction by adjusting cable lengths with PMX-2ED-SA 2-axis stepper motor controllers(Arcus Technology, Inc). These directions are denoted as the X* and Y* direction [not the same X and Y axes of manipulator space]; likewise, the cables on either side of the manipulator are correspondinglydenoted as the X* and Y* cables. To bend the manipulator in the X* direction, the X* cable is shortened until the desired bend is achieved. The Y* cable is left slack. A similar process is used to bend the cable in the Y* direction.

The manipulator also has two 10-bit analog inputs at its base on either side[3], which report the loads on the cables for both the X* and Y* directions.

Currently, the actions of all motors and so the movement of the manipulator may be fully controlled using keyboard commands via the MATLAB keystroke controller as the motors are interfaced with the PC via USB. An application may control the motors by invoking an ASCIIcommand set on the PMX and DMX C++ dlls.In this way, the keystroke controller also allows for custom motor commands to be sent to each motor and for force readings, positions readings, etc. to be polled from each motor as specified in the PMX and DMX user's guides (see Other Resources and Project Files). Finally, video is available via a camera mounted on the unit. However, the camera translates with the manipulator, compromising some of its utility.

In designing the interface between the PHANTOM and the manipulator, two different methods of control were considered. The first is based on positional data and allows the user to move the manipulator to a specific target location.

The second interface allows for continuous control of the manipulator via the PHANTOM. It relies on velocity data gathered from the PHANTOM for translation and rotation and positional data for bend.

The point/click interface is based on positional control between the PHANTOM and the manipulator. Essentially, the position of the PHANTOM, relative to a pre-defined origin, is passed into the control program, scaled, and then the manipulator is made to move to that target position. See below for a graphic representation.

Movement in this mode is achieved through the inverse kinematic model. Such a model allows the position of the PHANTOM Premium to be read as the desired position of the manipulator with some scaling. This desired position will then be the target position for the manipulator. Because of its motion, a natural way to model the workspace of the manipulator is as a cylinder. Thus, its movement can intuitively be written in cylindrical coordinates. In this scheme, translation is in the y-plane of the manipulator, rotation is in the x-z plane of the manipulator and about the y-axis, and the bend of the manipulator is

Rotation and translation were mapped by finding the number of motor pulses needed to translate/rotate a certain distance. These conversion factors were then tested repeatedly and updated with each trial. The final values predicted final position to sub-millimeter/sub-degree accuracy. These conversion factors are in the code and allow a desired movement to be translated into the necessary number of motor pulses to complete that movement.

Mapping bend in this way proved to be trickier because of the cable-driven actuation of the unit. During calibration, the configuration of the manipulator (i.e. x and y coordinates of the tip) was correlated with the number of motor pulses to achieve that configuration. Load data was also taken. Then, while the manipulator was operated, the number of motor pulses needed to achieve a desired configuration was known. Calibration was done experimentally by bending the manipulator a certain number of motor pulses with constant step size and tracking the position at each point using MATLAB’s ginput function which allows users to record mouse click positions. Using these points as a foundation, linear interpolation is used to find the number of motor pulses required to bend the manipulator to a certain configuration and thus achieve the target position. This also has the benefit of acting as a software stop on the bend of the manipulator. During calibration, the manipulator is not allowed to bend past 90 degrees, well within the safe bend range of the manipulator. Later, if a desired position requires bend outside the range measured during calibration, the number of motor pulses allowed is simply the maximum measured number of pulses.

A major issue with this control scheme is hysteresis—that is, the dependence of the system's current state on its past states. Because of cable slip that occurs during normal manipulator operation and other issues, the relationship between motor pulses and manipulator configuration is not constant. This leads to inaccuracy in the manipulator's motion to a target position. Another issue occurs in that cables must be replaced when they break. The variance in material properties and the length and tightness of the cable in the unit leads to inconsistencies between predictions based on calibration data and actual movement.

To correct this, various methods have been considered. These include also correlating configuration with the load read by the load cells at the manipulator tip. When cables slip, are tightened, or have to be replaced, their slack length changes–that is, the amount of cable that must be cycled through before it becomes taut and is able to bend the manipulator changes. However, load data can be used, rather than motor pulses, to predict end-effector position. As the relation between load and configuration seems to be constant over many trials and movements, this may serve as a reliable method of handling changing slack lengths.

Another method involves calibrating the manipulator prior to each session. This also leverages the fact that the relation between load and configuration is constant. Rather than performing the entire calibration process each session, the manipulator bends until the load cell reaches a certain threshold. The number of motor pulses required to reach this load is compared to the number of motor pulses required to reach the same load in the old calibration. A new calibration table is made that is offset by the difference in required motor pulses. Thus, prior to each subsequent session, the manipulator can automatically calibrate itself. Automating this prevents serious inconvenience to the user. The method of off-setting the motor pulse calibration rather than positioning based on load was chosen as calibrating on load would fail when the manipulator was contacted externally. To actually operate the manipulator in this mode, users move the PHANTOM stylus tip to a target position. They then click and release the button on the stylus to make the manipulator move to the target position (see Fig 6).

The continuous motion mode allows the user to continuously control the motion of the robot. Moving the PHANTOM stylus tip forward and back without the button pushed translates the manipulator forward and back. Rotating the PHANTOM stylus about a center point without the button pushed rotates the manipulator. The center point is defined as the point where the button was released. Finally, bending the manipulator is done by holding down the button and moving the PHANTOM stylus in the bend plane. See below for graphic representations of these.

Initially, position control was used for continuous motion control. However, because there is no on-the-fly position control in the software of our DMX motors for translation and rotation, this resulted in commands stacking up and being dumped or resulted in a jittery, stop-go movement. As a result, the manipulator was not as responsive or smooth as was desired. To remedy this problem, the PHANTOM was polled at varying rates instead of continuously. However, no optimal trade-off between responsiveness and smooth motion could be found.

As a result, velocity control was pursued rather than positional control. The PHANTOM can be polled for its velocity in the x, y, and z directions. Moreover, the DMX motors have functions that allow on-the-fly speed change, which precludes the problems with continuous positional control describe above. For translation, the velocity of the PHANTOM in its z direction was taken, scaled, and then fed to the motor controller, which then moves at the desired speed. The velocity was scaled through the function below:

As the function asymptotes at a pre-determined value, this function serves as a software limit on the velocity of the manipulator. Past a certain velocity, regardless of the actual PHANTOM velocity, the manipulator moves at some pre-determined safe max velocity.

Rotation proved to be more complicated than translation because the PHANTOM only reports linear velocities and angular velocity was needed to properly control the DMX rotation motor. An expression for angular velocity was found as the time derivative of θ. Again, θ is defined to be the angle of rotation from the positive x axis in the x-z plane, namely arctan(z/x). The expression for angular velocity is Clearly, angular velocity depends on x and z position and linear velocity. These values can be polled from the PHANTOM and so angular velocity can be found. Similar to translation, the below function was used to allow fine rotations and act as a software limit on max rotational velocity.

Because of the way the manipulator is rotated (rotating the PHANTOM about a central point), some intuitive way is needed to allow the user to keep track of that central point. Otherwise, the manipulator may act in unexpected ways. To help the user keep track of this information, the PHANTOM's spring force was used. The virtual spring is attached to the PHANTOM stylus at one end and the center point at the other. The spring constant was experimentally determined to allow the user to 'feel' the location of the central point without restricting the user's motion.

For both translation and rotation, a high-pass filter was built into the code to prevent unintended motion of the manipulator due to tremor or small movements of the hand.

Bend is controlled by PMX motors with the ability to change target position on-the-fly. This precludes the problems experienced with continuous positional control of translation and rotation. As a result, positional control was used to control bend in this interface. The PHANTOM workspace was divided into two parts via a line perpendicular to the bend plane so that if the tip of the stylus was to one side of the dividing line, the manipulator would bend to that side. Additionally, the farther the stylus was from the dividing line, the more the manipulator would bend. The value of the scaling was experimentally determined to allow for fine control of the manipulator bend.

When the manipulator is unrotated, this dividing line is simply a vertical line that passes through the origin of the PHANTOM workspace. Moving the stylus to left of the line bends to manipulator to the left and moving it to the right bends the manipulator to the right. To retain this sense of intuitiveness (i.e. the manipulator bends in the same direction that the PHANTOM is moved), the dividing line must rotate with the manipulator. Thus, if the manipulator has been rotated by 90°, the dividing line should now be horizontal so that moving the PHANTOM upwards bends the manipulator up and moving it downwards bends the manipulator down. To test which side of the threshold the PHANTOM tip is on, the following check is used. If the following expressions are satisfied, the X* cable is tightened:

The greater than sign is flipped for a rotation of greater than 180 degrees because this is the point at which the X* cable transitions from 'above' the threshold line to 'below' the threshold line. Initially, continuous control was in effect for bend, translation, and rotation all at the same time. While testing this interface, though, it was found that controlling all of these modes of motion at the same time was too complicated. It was especially difficult to differentiate between bend and rotation. Thus, bend was decoupled from the other modes of motion to give the user more control over the manipulator's movement. This is done by only sending commands to the DMX motors for translation and rotation while the stylus button is not pressed. While the stylus button is pressed, commands are instead sent to the PMX motors to control bend. An important factor to consider was what to do with the bend of the manipulator when the button was released. It was deemed most intuitive to have the manipulator retain the bend while being rotated and translated. To ensure that the bend can then be modified from that position once rotation/translation is complete, the position of the manipulator tip is stored as a vector whose magnitude depends on the bend of the manipulator. This vector is translated and rotated along with the motion of the manipulator to store the position accurately as the manipulator moves.

Additionally, in response to feedback from users, translation and rotation were decoupled to allow easier control of the PHANTOM. To achieve this, the absolute value of the scaled translational and rotational velocities are compared. The minimum of the two values is set to zero, and this information is sent to the motor controller. Since translational and rotational velocities are not directly comparable, the values must be rescaled. Experimentally, it was found that doubling the rotational velocity results in proper comparisons leading to the desired motion of the manipulator. The idea behind this is that the user will move more quickly in the way of the desired motion. Thus, the unintended motion should have a lower velocity than the desired one. This essentially allows the user to ‘select’ which mode of motion he wants simply by moving the PHANTOM in the appropriate way.

Because bend is controlled by two cables, each actuated by a different motor, changing the direction of the bend takes a relatively large amount of time. For example, if the manipulator is bent to the left, and we want to bend it to the right, the left cable must first be slacked. Then the right cable can be tightened to achieve the desired amount of bend. The left cable must be slacked before the right cable is tightened to prevent both cables from being tight at the same time. At best, this will result in a great deal of cable slip, and at worst, it will cause a cable to snap. Both are undesirable in operation of the manipulator.

To compensate for this delay and to increase the responsiveness of the manipulator in bending, the velocity of the motors was adjusted depending on whether they are tightening the cable or slacking it. If a motor is slacking the cable, it moves very quickly (1200 pulses/sec) while if it is tightening the cable, it moves more slowly (700 pulses/sec). This difference in velocities is instituted to prevent one cable from being tightened before the other cable is slacked. Again, this could result in major cable slip or a cable snapping. Experimentally, this was found to significantly increase the responsiveness of the manipulator.

The OpenHaptics Toolkit is an architecture that allows users to develop applications for haptic devices such as the PHANTOM Premium. The API is implemented in the C++ programming language and allows users to interact with the device by creating and displaying environment geometries, setting force and stiffness parameters, and setting up callback responses to interactions. The motor controllers also provide libraries that an application can use to interface with the motors rather than sending ASCII commands directly to the PMX and DMX motors to set up a serial port connection. The existing MATLAB keystroke controller creates an object that can then call the motor controller dll to control the manipulator through a GUI. This dll can also be utilized to send commands to the motors based on input from the PHANTOM. We propose two different approaches to creating an interface integrating the PHANTOM and the motor controllers.

Our first option was to port, or convert, the existing MATLAB controller to C++ and then execute PHANTOM calls from within the same C++ environment. To accomplish this, we planned to build upon an existing C++ GUI developed by APL for the PMX control using the Qt GUI development framework. After this was complete, we planned to port our inverse kinematics model code, along with other relevant code, from MATLAB to C++ as well. Former students have developed a simplified forward kinematics model of the manipulator using RobWorks, a framework for simulation and control of robotics, also written in C++, which we hoped to incorporate into our program in this option as part of the visualization.

Our second option was to write the PHANTOM controller in C++ and create a pointer to a MATLAB engine session within the program to allow us to pass variables and commands from C++ to MATLAB. MATLAB provides a library that allows C or C++ programs to start a non-graphical (GUI-less) MATLAB process for using MATLAB as a computational engine. This library includes functions to either directly execute command strings in MATLAB or to pass buffers back and forth between C/C++ and MATLAB. This method would build upon the existing keystroke controller and would allow us to use the inverse kinematics code that we would develop in MATLAB.

We experimented with both options and began by porting the keystroke controller to C++. We successfully built and integrated the Johns Hopkins CISST libraries into the Qt program and were able to initialize and control all axes of the robot via the controller.

Concurrently, we developed and prototyped the inverse kinematics model using MATLAB for the data collection and calculations. We built upon the MATLAB keystroke controller, which contained a rich set of functionalities developed by APL. These include functions for rotation calibration, camera calibration and display, and bump tests for finding the hard limits of the motors. To allow for full utilization of the keystroke controller, we decided to use option 2 (calling MATLAB from C++). This also allowed us to save time by avoiding having to port the inverse kinematics and bend calibration functions we developed in MATLAB to C++. This proved to be a sound choice as it granted us faster turnover for prototyping and testing our algorithms.

In its current state, the program accepts as input the control mode of the manipulator (1 for point/click, 2 for continuous control). It initializes the connection to the PHANTOM to first set up a virtual scheme in which the tip of the PHANTOM is represented as the tip of cone. It then creates a MATLAB process, creates a MATLAB keystroke controller object, displays the keystroke control GUI, and initializes the robot by going through a bend calibration routine and bump tests for bend and translation motors to find the hard limits. Then, depending on the control mode, the program executes the appropriate callback functions for the press and the release of the button on the PHANTOM joystick. It logs position data from the PHANTOM and manipulator throughout the course of the program execution. When the program exists, the manipulator resets to a predetermined, constant home position. For more details, please refer to the reference manual (see User Guide).

Controlling manipulator using force-feedback integrated PHANTOM®

1. Turn on hardware (manipulator, PHANTOM®)
2. Specify control mode (position control, velocity control)
3. Start program
4. User feels spring force relative to current coordinate system origin
5. If position control:
   1. Hold down PHANTOM® button to set new origin
   2. Move to desired absolute position relative to new origin
   3. Release PHANTOM® button to move manipulator
6. If velocity control:
   1. Click and release PHANTOM® button to begin
   2. To bend: hold down button while moving in bend directions (as indicated by arrows)
   3. To translate: move joystick towards desired direction
   4. To rotate: make circle using steady movements in desired direction
7. End program
8. Turn off hardware

We explored the use of visual cues for giving the user a better feel of his or her position within the lesion. We experimented with two methods: an live image overlay and a software simulation. The first method involved overlaying a marker on the live video feed that tracked the calculated position of the tip. To provide a sense of depth for this birds-eye-view, we scaled the size of the marker according to the manipulator tip's distance from the camera lens. A bent manipulator rotated 90 degrees so that the tip faced up, for example, would generate a larger marker than a perfectly straight manipulator.

While useful for validating our kinematics model, this method would not be employed in a true surgical setting for a THA procedure, since we would not be able to obtain a video feed of the inside of a lesion. Therefore, we also implemented a second option. This visualization tracked the position of the tip from a birds-eye-view as well, but instead displayed the marker's relation to the environment by showing three fixed markers in the workspace, which represented the positions of the posts of different heights used for our trials. Rather than using a dot of varying size, we represented the height (z position) of the tip by a separate adjacent window that displayed a marker whose height varied between the minimum and maximum of the manipulator bend radius. This implementation provided both more speed than the video overlay, which experienced a small lag, and more flexibility, since it is not line-of-sight dependent. Also, if we change the workspace setup in the future, we may swap in a 3D CAD model of the working environment.

The three posts are the posts of the trial phantom and the trace at the bottom is the manipulator tip path.

We explored alternative forms of sensory substitution for communicating force information from the manipulator tip to the user. In teleoperated procedures for dexterous surgical procedures such as suturing, the use of sound has proven to increase both accuracy and precision when complete, fully realistic haptic feedback is not available [5].

Our primary challenge was to communicate the interaction forces felt at the manipulator tip to the user. In order to accurately display end effector forces for truly realistic teleoperation, we require at least six DOF for position and orientation and additional task-specific DOFs for tasks such as gripping. We currently have no force sensors at the tip of the manipulator, only force sensors at the base of the manipulator that measure the load on each of the cables. Therefore, any force information may only be inferred based on the geometry of the cables.

We developed a scheme to infer the in-bend contact forces by comparing the measured cable load cell forces to the estimated forces from pre-recorded trials. Therefore, we would like to make a distinction between end-effector force feedback and our method using cable displacement force feedback. It is able to detect forces that result in changes in cable tension, such as pushing the manipulator towards or against its direction of bend when the cable is not slack (when the manipulator is bent). We determined a threshold experimentally below which the system will not emit a sound to account for jitter. Above the threshold, the program will emit beeps at a frequency directly proportional to the force displacement.

To compare the efficacy and intuitiveness of the test mappings, trials (IRB approval obtained on 4 April 2012 - see Other Resources and Project Files) were conducted on subjects inexperienced with the PHANTOM Premium. They used the PHANTOM to control the manipulator. Trials involved 6 subjects with no experience in any kind of surgery. The idea is to gauge how easily each subject can learn to use the interface effectively, allowing us to measure how intuitive each interface is.

The above apparatus was designed using Autodesk Inventor and then 3D printed. On each post is a colored marker in a different orientation and at a different height.

Subjects were asked to move the PHANTOM to cause the manipulator to touch the colored markers in a given order. The time it took for the subject to perform this task was recorded. Subjects were then asked to repeat the task twice for each interface: keyboard control, point/click, and continuous control. The times were compared across interfaces.

The above figure shows that the keyboard actually seems to be the most efficient method of controlling the PHANTOM, followed by continuous control and then point-click. However, this ordering is well within the rather large standard deviations of each set of data, indicated by the error bars. Further trials are necessary to better understand the efficaciousness of the different interfaces. Further, during testing, many of the trials were interrupted by program bugs that caused motors to stop moving or to move in unexpected ways. Other possible explanations for this discrepancy are poor instructions on how to operate in a given interface and also that users are likely much more experienced using a keyboard than with a PHANTOM haptic controller.

Qualitative feedback obtained from users after the trials has allowed for improvements to be made to each interface. These include decoupling translation from rotation for the continuous control interface. In addition, for the point/click interface, the desired position of the manipulator was originally based on the final position (judged when the button was released) of the PHANTOM relative to its starting position (judged when the button was pressed down). Now, the target position of the manipulator is based on the absolute location of the PHANTOM when the button is pressed. The absolute coordinate system has its origin at the position where the PHANTOM tip was when the stylus button was first pressed after the session was initialized. Users also suggested a function for the continuous motion interface that allows the position tracking and manipulator movement to be paused. These, and other refinements, call for further trials to be done.

In addition to times, load data, positional data, and motor pulse data were collected from the manipulator, and positional and velocity data were collected from the PHANTOM throughout the trials. Using these, trajectories and planned paths of the users were determined, analyzed, and compared between interfaces. Two corresponding trajectories are shown below.

Note that the general motions indicated by these trajectories is the same. The manipulator is first bent, then translated and then rotated before being bent further and then unbent. The scaling between the PHANTOM and manipulator workspaces is clear, especially for rotation. In addition to inexperienced user trials, we had also hoped to have a surgeon, Dr. Mears, use our system and provide us with qualitative feedback. However, due to availability issues, it was not possible to do so prior to the poster presentation date.

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentations
  • Manish Mehta
    • The Benefit of Force Feedback in Surgery: Examination of Blunt Dissection (Wagner et al.)
      • Presentation (Wagner et al.)
      • Report (Wagner et al.)
  • Piyush Poddar
    • Design of a New Cable-Driven Manipulator with a Large Open Lumen: Preliminary Applications in the Minimally-Invasive Removal of Osteolysis (Kutzer et al.)
      • Presentation (Kutzer et al.)
      • Report (Kutzer et al.)
  • Jessie Young
    • Methods for Haptic Feedback in Teleoperated Robot-Assisted Surgery (Okamura)
      • Presentation (Okamura)
      • Report (Okamura)
• Project Final Presentation
  • Final Poster
  • Teaser
• Project Final Report
  • Final Report
  • User Guide

[1] W.P. Liu, B.C. Lucas, K. Guerin, and E. Plaku, Sensor and Sampling-based Motion Planning for Minimally Invasive Robotic Exploration of Osteolytic Lesions, in Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, Sept. 2011.

[2] S.M. Segreti, M.D.M. Kutzer, R.J. Murphy, and M. Armand. Cable Length Prediction for a Compliant Surgical Manipulator, in Proceedings of the 2012 IEEE International Conference on Robotics and Automation, May 2012, in Press.

[3] M.D.M. Kutzer, S.M. Segreti, C.Y. Brown, R.H. Taylor, S.C. Mears, and M. Armand, Design of a new cable driven manipulator with a large open lumen: Preliminary applications in the minimally-invasive removal of osteolysis, in Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA2011), May 2011, pp. 2913-2920.

[4] B. T. Bethea, A.M. Okamura, M. Kitagawa, T.P. Fitton, S.M. Cattaneo, V.L. Gott, W.A. Baumgartner, and D.D. Yuh. Application of Haptic Feedback to Robotic Surgery. in Journal of Laparoendoscopic Advanced Surgical Techniques 14.3, 2004, pp. 191-95.

[5] A.M. Okamura, Methods for Haptic Feedback in Teleoperated Robot-assisted Surgery. in Industrial Robot: An International Journal 31.6, 2004, pp. 499-508.

[6] C.R. Wagner, N. Stylopoulus, and R.D. Howe, The Role of Force Feedback in Surgery: Analysis of Blunt Dissection, Proc. Haptics Symp., Mar. 2002.

[7] G. Tholey , G.P. Desai, A.E. Castellanos, Force feedback plays a significant role in minimally invasive surgery - results and analysis, Ann of Surg, 2005;241(1): pp. 102–109.

[8] O.A.J. Meijden, M. P. Schijven, The Value of Haptic Feedback in Conventional and Robot-assisted Minimal Invasive Surgery and Virtual Reality Training: A Current Review, Surgical Endoscopy, 2009, pp. 1180-190.

[9] C.R. Wagner, R. D. Howe. Mechanisms of Performance Enhancement with Force Feedback, WHC '05 Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2005, pp. 21-29.

1. CISST libraries and Surgical Assistant Workstation (SAW)
2. PHANTOM Premium Documentation
3. DMX Motor Documentation
4. PMX Motor Documentation
5. IRB Forms
   • IRB Approval
   • IRB Application
   • Consent Form
   • Recruitment Materials
   • User Survey