Optimization-Based Virtual Fixtures for Robotic Systems

Most robotic assisted surgical procedures are characterized by restricted access to the workspace as well as constrained manipulation of a surgical tool. In such cases, the surgeons’ ability can be augmented by techniques such as virtual fixtures. Virtual fixtures are algorithms which provide anisotropic behavior to surgeons’ motion commands in addition to filtering out tremor to provide safety and precision.

Typically, surgical tasks have a certain degree of uncertainty that arises from factors such as registration errors, variations in anatomy and changes during procedures. Consider an example task of placing a surgical tool at a point in space. Depending on the nature of the procedure, one can define a region and tool placement within this region that would lead to the expected outcome. This region could be on the order of a few microns for retinal vein cannulation or hundreds of microns for a biopsy procedure. Furthermore, we can define another region where the surgeon might deliberately want to place the instrument to account for some uncertainties inherent in surgical procedures. In other words, we would like to have some compliance in the virtual fixture, while maintaining a preferred motion. Therefore, we define 3 different regions:

  • Preferred region: This region defines the expected outcome.
  • Safety region: The tool could temporarily be in this region for fulfilling some expected task.
  • Forbidden region: The tool never could be here for safety purposes.

The relationship of these three regions depends on the surgical task.

Project Goal

For this project we formalize a library of virtual fixtures for task primitives. Our paradigm covers the implementation of guidance virtual fixtures, and forbidden region virtual fixtures, with both “hard” and “soft” constraints. Our approach uses a weighted, multi-objective optimization framework to solve for incremental joint motion given the instantaneous kinematics of the manipulator and the geometric constraints. Note that we treat the robot as a purely kinematic device, independent of manipulator characteristic: teleoperative or cooperative controlled; admittance or impedance type.


We formulate the motion control of the robot as a quadratic optimization problem, in which constraints and optimization criteria are combined from multiple sources. These include:

  • Joint limits and other robot kinematic constraints.
  • Surgeon commands from a master hand controller.
  • Real vision or other sensor data.
  • Descriptions of desired behavior built up from simple primitives.
  • Registered anatomic models of the patient.

We have applied this formulation to teleoperation of complex robots such as snake robots, assistance in tasks such as suturing, implementation of safety regions in neurosurgery, alignment aids in targeting tasks, and retinal surgery. We solve these problems at interactive rates (typically, from 20 Hz to 200 Hz, depending on problem size, using either a linearized constrained least squares solver based on the work of Lawson and Hanson or a more general solver, depending on the problem.

Future Plan

  • Redesign for use with daVinci
  • Integration into cisst

Project Personnel

  • Russell Taylor
  • Ankur Kapoor
  • Ming Li


  • J. Funda, R. Taylor, B. Eldridge, S. Gomory, and K. Gruben, “Constrained Cartesian motion control for teleoperated surgical robots”, IEEE Transactions on Robotics and Automation, vol. 12- 3, pp. 453-466, 1996.
  • R. Kumar, A. Barnes, G. Hager, P. Jensen, and R. Taylor, “Application of Task-Level Augmentation for Cooperative Fine Manipulation Tasks in Surgery”, in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2001, Utrecht, October, 2001. pp. 1417-1418.
  • M. Li and R. Taylor, “Optimal Robot Control for 3D Virtual Fixture in Contrained ENT Surgery”, in Proceedings of the Sixth International Conference on Medical Image Computing and Computer Assisted Intervention – MICCAI 2003, Montreal, Nov 15-18, 2003. pp. 165-172.
  • M. Li and R. H. Taylor, “Spatial Motion Constraints in Medical Robots Using Virtual Fixtures Generated by Anatomy”, in IEEE Conf. on Robotics and Automation, New Orleans, April, 2004. pp. 1270-1275.
  • A. Kapoor, M. Li, and R. H. Taylor “Spatial Motion Constraints for Robot Assisted Suturing using Virtual Fixtures”, in MICCAI, Palm Springs, CA, 2005. pp. 89-96. PMID: 16685947
  • A. Kapoor, N. Simaan, and R. H. Taylor “Suturing in Confined Spaces: Constrained Motion Control of a Hybrid 8-DOF Robot”, in International Conference on Advanced Robotics, Seattle, WA, July 17-20, 2005. pp. 452-459.
  • A. Kapoor, M. Li, and R. H. Taylor “Constrained Control for Surgical Assistant Robots”, in IEEE Int. Conference on Robotics and Automation, Orlando, May 15-19, 2006. pp. 231-236.
  • A. Kapoor, N. Simaan, and R. H. Taylor, “Telemanipulation of Snake-Like Robots for Minimally Invasive Surgery of the Upper Airway”, in MICCAI Medical Robotics Workshop, Copenhagen, October, 2006. pp. 17-25.
  • M. Li, M. Ishii, and R. H. Taylor, “Spatial Motion Constraints in Medical Robot Using Virtual Fixtures Generated by Anatomy”, IEEE Transactions on Robotics, vol. 23- 1, pp. 4-19, 2007.
  • M. Li, A. Kapoor, and R. H. Taylor, “Telerobot Control by Virtual Fixtures for Surgical Applications”. in Advances in Telerobotics Human Interfaces, Bilateral Control and Applications, M. Ferre, M. Buss, R. Aracil, C. Melchiorri, and C. Balaguer, Eds.: Springer, 2007, pp. 381-401.
research/optimization_vf.txt · Last modified: 2019/08/07 12:01 (external edit)