Last updated: 5/10/2023 at 3:55 PM

The outcome of this project is a new functional frequency domain-based admittance controller that optimizes the transparency of the current Galen robot system in the mock OR while maintaining system stability. This will be accomplished by re-imagining the current control system entirely with basic linear models and then introducing an admittance controller in conjunction with filtering or any of linear control shaping to obtain a controller that better compensates for stability issues that are presently experienced when forces are applied to the robot by the user and especially the environment. Furthermore, the controller will be designed and optimized with hand over hand transparency in mind.

• Students: Brevin Banks
• Mentor(s): Ugur Tumerdem, Manish Sahu, Adnan Munawar, Mohammad Salehizadeh, Russell Taylor

https://www.galenrobotics.com/

The current Galen campus robot controller faces performance and stability issues when forces are applied and the robot in general feels heavy and hard to move. When using the robot for hand over hand assistance in a micro-surgical setting, stability is critical to prevent accident and loss of precision, and improving the quality of the overall controller feeling for the robot makes the user more comfortable and happy with the robot interactoion. Thus, in addition to controller stability improvement, we want to optimize the transparency with a new controller. Transparency in this case is a sort of hand over hand mechanical transparency in that surgeons should not feel the dynamics of the robot or the control system during free motion and should only feel the contact and constraint forces. Unfortunately, pure transparency is not achievable because of frictional forces and inertia but much is still to be optimized in terms of robot dynamics in transparency. Stability innately affects the transparency of the control system and cannot be easily adjusted by simply tuning gains. There is much work to be done in the field of transparency optimization.

The significance of this work is found in the quality of the resultant controller. This would provide the Galen with a controller that allows for more natural and steady movements in the operating room. The controller will be designed to allow for easy application of Virtual fixtures. The controller will allow surgeons to learn and procedures and adapt to operations quicker and perform more accurately.

Updates to the Deliverables have been made to justify the project scope for the project semester period. The user study has been set aside as less critical and will be addressed at a later term.

• Minimum: (2/13/2023 - 4/18/2023) PREVIOUS DATES: (2/13/2023 - 4/3/2023)
  1. A stable frequency domain admittance controller in MATLAB code for the Galen surgical robot in the AMBF simulation environment
  2. A simulated control system package with optimized transparency containing code files that can be used in AMBF and MATLAB and is transferable to hardware

• Expected: UPDATED EXPECTED DATES: (4/7/2023 - 4/25/2023) PREVIOUS DATES: (2/20/2023 - 4/3/2023)
  1. An applied control system package to the Mid-level Galen controller that is stable and optimized for transparency including virtual fixtures
  2. A validated objective improvement in transparency according to specified metrics, with identified system dynamics

UPDATED MAXIMUM DELIVERABLE

• Maximum: (4/25/2023 - 5/5/2023)
  1. Implementation of Virtual Fixtures that constrain the robot hardware within desired simulated bounds.

EXTENDED MAXIMUM DELIVERABLE

• Maximum: (Expected Summer Operations)
  1. A completed user study to investigate subjective transparency preference with different controllers on the Galen robot

Key Design Elements

To understand the goal and approach of this project, it is important to understand what stability, transparency, and admittance controllers are. Stability can be best explained through the example of operation. We see stability in interactions of the system input (e.g., the operator’s hand and the contact between tools and the patient or environment) and the response of the robot dynamics from the hardware. The user inputs some type of desired force, Farm(s), and then a controller, C(s), interprets this force and outputs it into a form the robot dynamics, also called the plant, P(s), can use to move the hardware to the desired location. When certain inputs cause the robot to behave erratically, we observe instability in the dynamics of the robot such as uncontrolled shaking or fast, dangerous movements. To ensure that the system is stable we often apply feedback and different elements to our controller, C(s), that will keep the output of the dynamics within safe stability margins.Stability can be observed quantitatively and qualitatively through analysis of things such as bode plots’ relative phase and gain margins, left hand vs right hand system poles, and the difference between the number of poles and loops of a Nyquist plot. When a control designer simply tunes system gains, filters, integrators, etc. they can simulate the effects in something such as MATLAB SIMULINK which allows them to quickly see how their changes affect the system performance and stability.

Transparency and stability are closely related in terms of the performance of a control system. Transparency is a measure of how easy the system is to manipulate or drive in the desired direction without feeling heavy resistance or drag. Transparency is reflected in system stability and the lack of impedance from the robot during performance. By observing the impedance in hand with stability we can infer the relative transparency of the system. Therefore we will make the assumption that as we tune for stability and change some metrics about the system impedance with our controller we will be able to optimize the transparency. This approach is known as the design of an admittance controller and the fundamental design our control system.

Controller Design

Admittance is the inverse of impedance. This approach will take a basic velocity control based PID controller, C(s), for an assumed linear system, P(s) and apply an integrator with a gain after the input before the controller to virtually remove system impedance. We’ll call this the admittance controller A(s). This will make the system seem ‘transparent’ because the resistance of the dynamics are compensated for, but it is important to know that if the system impedance is completely removed the robot can be easily tripped unstable. It would be essentially like dividing by 0. Moreover, it is not physically possible to compensate for all the inertia and resistance effects of the robot in the real world. There will be some level acceptable resistance felt by the operator. All in all, particular care must be taken when tuning the gain of A(s). The goal would be to minimize the impedance and conversely maximize admittance.

Also note the inclusion of external impedances from the environment and the human arm. Because there are uncertainties in the operation of the robot, we will include these impedances and optimize the transparency for a range of different human and environment impedances. Simply put, based on data in literature, human and environment impedance can be modeled as a mass-spring-damper with each their own m, b, and k variables that can be tuned. Upper bound values for m, b, and k have been obtained for both the human arm (mh = 5kg, bh = 41 Ns/m, kh = 401N/m) and environment and (me = 0kg, be = 0Ns/m, ke = 16599N/m) with the lower bound being equal to 0 for all metrics. See the work of Aydin et.al. (2020) for the framework of this impedance approach.

Most Robots are highly nonlinear, and their plants are difficult to accurately model. It is valid to be concerned about assuming the plant is linear. To address this concern, once the controller has been verified to work in simulation with a basic mass-spring-damper plant for velocity control (i.e. P(s)=(ms^2)/(ms^2+bs+k) ) we will perform some type of dynamics identification either with linear-time-invariant (LTI) frequency based identification or other acceptable linear approximations.

Another detail to note about the control architecture is the filter/loop-shaper block, H(s), which will be used in fine tuning of system poles, phase and gain margins, and filtering if necessary. Ideally, blocks A(s) and H(s) would be the two most critical blocks for tuning regarding the stability and impedance of the system, with the external impedances Zh(s) and Ze(s) present to simply give us an understanding of the affects of interactions on the system.

The controller will be designed and built in MATLAB and MATLAB SIMULINK, and virtual optimization of the controller for transparency and stability will be performed there. Once the controller has been verified to perform adequately in MATLAB simulation, it will be used in conjunction with the Asynchronous Multi-Body Framework (AMBF) environment. The AMBF environment enables the user to see how a virtual robot can interact with the environment and with the implementation of different controllers a virtual representation of the robot performance will be shown. Upon tuning the controller in MATLAB and AMBF the controller will be packaged and structured to accept the inputs and outputs of the real Galen Robot hardware.

A sub task of the controller design is updating the dynamics it will be tested on. For the AMBF simulation it is critical to have a fair model of the Galen Robot available to accurately measure the performance of the controller. Currently a Galen Robot model is available, but the virtual dynamics are not correct. As such the dynamics of the robot need to determined using basic robotics dynamics based on the robot structure and joint type. The forward and inverse kinematics will be determined and a simple low-level controller will be constructed to allow for velocity control of the joint space.

After the dynamics and controller work synergistically, it will be possible to implement virtual fixtures in the control scheme. Methods for the implementation of these have not yet been discussed by the team, but many acceptable and simple methods exist that are reliable for linear models that can be quickly tested. Low Level Galen Integration If the hardware and software interfaces are available for integration, then the controller package will be uploaded and the anticipated transparency and stability of the controller will be observed. Testing on the real-world controller would ideally be intermittently and not only after the simulation is fully complete. Due to the time constraints of the project and the nature for the need of the simulated design to match up with the real-world interface design constant testing will be critical. It is expected that there will be much to refine in the controller due to the nature of real-world dynamics, so the team will return and update A(s), H(s), or other necessary metrics until real world stability and transparency are achieved.

Work Flow

The code for simulation will be adequately tested in MATLAB and AMBF and intermittently the controller will pushed to the real hardware if available. When completed in simulation, full efforts with the controller will be on the hardware. Simultaneously, necessary dynamics in simulation will be updated as time permits, and documentation for all code updates, software changes, and project changes will be generated and organized. The controller package on the physical hardware will be deemed completed for this project when it is A) proven stable on the physical hardware. B) a clear measurable metric of transparency is obtained and observed on the physical hardware. C) an example of a virtual fixture has been implemented and functions as expected. D) the stability and transparency perform better than the current controller does in its presently unstable conditions.

User Study (THIS HAS BEEN MOVED OUT OF THIS SEMESTER'S SCOPE, THE IMPLEMENTATION OF VIRTUAL FIXTURES HAS BECOME PRIORITY FOR THE MAXIMUM DELIVERABLE)

Should time allow, a user study will be carried out where local surgeons will be recruited to qualitatively report on controller preference. Several tests will be designed that have the surgeons run through basic performance tasks. The surgeon would be running these tasks on the original Galen Robot control system and our new control system. They would then be given surveys on which system they preferred according to several different metrics. This user study would allow future work to reanalyze the effectiveness of a frequency-based admittance controller design and open way to potentially more applications of this control scheme or refinement of the current approach.

* As of 5/9/2023

1. Milestone name: Connect and Familiarize with AMBF and MATLAB Environment for Simulation
   • Planned Date: 2/13
   • Expected Date: 2/27
   • Status: Done
2. Milestone name: Update Simulated Galen Robot Dynamics
   • Planned Date: 2/27
   • Expected Date: 3/13
   • Status: Done
3. Milestone name: Brainstorm Methods for Implementing Simple Admittance Controller and Choose Transparency Metrics
   • Planned Date: 2/13
   • Expected Date: 2/27
   • Status: Done
4. Milestone name: Implement a Linear Controller Modeling the Galen as Mass Spring Damper in MATLAB Simulink Simulation
   • Planned Date: 2/27
   • Expected Date: 3/6
   • Status: Done
5. Milestone name: Assess and Optimize Transparency and Stability of Controller in MATLAB
   • Planned Date: 3/6
   • Expected Date: 3/20
   • Status: Done
6. Milestone name: Identify the System. Convert the Controller Plant to Robot dynamics, Update, and apply to AMBF Simulation
   • Planned Date: 3/20
   • Expected Date: 4/12
   • Status: Done
7. Milestone name: Add Virtual Fixture to the Controller in AMBF
   • Planned Date: 4/10
   • Expected Date: 5/18
   • Status: In Progress
8. Milestone name: Apply Simulated Controller to Real Galen Hardware
   • Planned Date: 4/7
   • Expected Date: 5/25
   • Status: Mostly Completed

NEW TASKS FOR MODIFIED MAXIMUM DELIVERABLE

1. Milestone name: Implement Virtual Fixtures on Real Galen Hardware
   • Planned Date: 4/25
   • Expected Date: 6/5
   • Status: In Progress

TASKS ON HOLD FOR FUTURE TERM

1. Milestone name: Design User Study for Testing New Galen Controller with Respect to the Current Controller and Obtain IRB Approval
   • Planned Date: 5/5
   • PREVIOUS Expected Date: 6/2
   • Status: ON HOLD
2. Milestone name: Recruit Users and Perform Testing
   • Planned Date: 6/2
   • PREVIOUS Expected Date: 6/16
   • Status: ON HOLD
3. Milestone name: Analyze Data and Conduct Further Research/Write Paper
   • Planned Date: 6/16
   • PREVIOUS Expected Date: 7/6
   • Status: ON HOLD

Galen Controller Transparency Optimization Project Required Tasks and Task Reliance Most tasks for the project are divided into 4 groups, each of which is primarily dependent on the previous group of activities completion. Groups 1 and 2 can be done independently if a MATLAB Installation with Simulink and the correct environment tools are available to start group 2 tasks. Group 3 is dependent on groups 1 and 2’s completion. Group 4, if time allows, is dependent on the completion of Group 3. For an expected completion time frame for each task see “Galen Transparency Control Gantt Chart.xlsx”. Note: an extra group is also present in the Gantt Chart for CIS II deliverables and is not listed here for the sake of product relevance.

Group 1: (Tasks 1.1 done in Parallel with tasks 2.1)

Task 1.1) Connect and Familiarize with AMBF and MATLAB Environment for Simulation (1 Week) Install Software, Load in Current Galen Robot

Task 1.2) Update Simulated Galen Robot Dynamics (1 Week) Observe Real World Dynamics and Add More Realistic Model to Simulation

Group 2:

Task 2.1) Brainstorm Methods for Implementing Simple Admittance Controller and Choose Transparency Metrics (1 Week) Develop Basic Idea of Key Attributes Required for Controller Such as Control Diagram and Key Concepts

Task 2.2) Implement a Linear Controller Modeling the Galen as Mass Spring Damper in MATLAB Simulink Simulation (1 Week) MATLAB Simulink Simulations for Control Systems using a Primitive Linear Model

Task 2.3) Assess and Optimize Transparency and Stability of Controller in MATLAB (2 Weeks) Explore and Adjust Admittance, Frequency Filter, Feed Forward, Nyquist Plot and Frequency Plots

Task 2.4) Identify the System. Convert the Controller Plant to Robot dynamics, Update, and apply to AMBF Simulation (2 Weeks) Change the Dynamic Model and DOF to be More Realistic of the Galen Robot, Perform Simulations in AMBF

Task 2.5) Add Virtual Fixture to the Controller in AMBF (1 Week) Apply Virtual Constraints or a Nonlinear Approach to the Control Scheme

Group 3: (Tasks 3.1 on going during all of tasks in group 2)

Task 3.1) Apply Simulated Controller to Real Galen Hardware (4 Weeks) Upload and Update Simulated Controller to be compatible and Safe with Real Hardware

UPDATED TASKS:

Task 3.3) Implement Virtual Fixtures on Real Galen Hardware (2 Weeks) Create Virtual Bounding method, Repel Force In Opposite Direction, Detect Collisions, Debug Issues with the Controller on Hardware, Evaluate the Performance of the Controller on the Galen

GROUP 4 TASKS CURRENTLY ON HOLD UNTIL A LATER TERM

Group 4: (Task 4.1 will require an IRB and that will be an ongoing process beginning February 8th. The Application should be submitted by Feb 24th at the latest)

Task 4.1) Design User Study for Testing New Galen Controller with Respect to the Current Controller and Obtain IRB Approval (1 Week) Design Tests, Data to collect, Number of Necessary Users

Task 4.2) Recruit Users and Perform Testing (2 Weeks) Advertise and Find People to Assess Controller Performance

Task 4.3) Analyze Data and Conduct Further Research/Write Paper (1 Week/Post Semester) Evaluate Collected Data, Find Story, Significance, and Application of Work, Aim to Publish and Write-Up Work.

Students:

Brevin - Core Research Student

Mentors:

Ugur - primary controls mentor

Mohammed - Galen operation and System Identification mentor

Adnan - AMBF and Galen simulated controls mentor

Manish - Matlab Comms and ROS Comms mentor

Russel Taylor - Supporting Mentor

Meetings: Weekly: Mondays at 1:30PM

Source Control:

Git_hub Link: https://gitfront.io/r/user-6241796/DtaCrgWCApHi/CIS2-Admittance-Control/

Documents are developed, named, and kept according to the standards outlined in TOGAC2_D_DocumentControl.xlsx

In summary documents are named after TOGAC - “Transparency Optimization of the Galen Surgical System with a Frequency Domain Admittance Controller Design project documents” - Which represents all documents related to the controller development.

The number following TOGAC is the order of the creation of that document among all project related documents.

Other tags follow according to relevance within project tasks (AMBF simulation docs, MSD Mass Spring Damper simulation docs, RGC Real robot implementation docs).

Final tags represent the type of document. R for reports, CD for controller design documents, D for supporting development documents, IN for developed instructions, and P for presentations.

The first step in the controller development was feasibility of the current approach. From the work of [6] and [2] our efforts were primarily focused on reproducing the already obtained results and comparing the method with our needs. [6] identified a typical mass spring damper system for use with an admittance controller. Thier mass spring damper was replicated along with their controller and was tested. The desired admittance Yv was used to make the controller transparent, and the observed admittance on the human was verified to agree with the results reported in the paper. We found that they did agree, and we identified methods in which we could further refine and shape our admittance controller with the methods from this paper. The resulting figures and an account of the findings are here.

TOGAC11_MSD_R_AdmitConFeedbackandForwardVsFeedback

This paper helped us consider admittance with a feedforward control adaptation that could improve the transparency. Further analysis with this concept was incorrporated in other testing we've performed, but ultimately note that we did not choose to include this feedforward control style in our formal design. Many reasons for this include that the feedforward design removes much of the integrity of the internal stability, and the capturing of plant dynamics using linear system identification methods made incorporating such a scheme rather difficult in the real world. A formal feedback admittance controller design was made as shown in the technical approach section. A feedforward design was also produced as can be found in the design documentation here below. TOGAC27_CD_TransferFunctionDerivAndStability

With the given techniques from [6], we observed the techniques of the stability analysis in [2]. Mirroring stability and cost maps from [2]'s figures 6 through 9 were generated for a fixed mass spring damper system we arbitrarily designed. These pseudo plants took the form of fixed damping and stiff mass spring dampers. Transfer functions for these systems were obtained and they were tested for stability. The resulting tests produced the results found in this report. TOGAC17_MSD_R_StabilityFBandFFApproach The conclusions were to use the feedback design and the methods from [2] to implement our controller.

With the tool from [9], [10], and [11], known as the Asynchronous multi-body framework (AMBF) simulation environment, useful for controlling, observing, and operating a robot in a highly adaptive environment with multiple process running, we began building and applying our controller to a 3D model of the Galen robot. The AMBF environment can account for robot control along side simulated collision detection coupled with the live control and potential teleoperation of robots and robotic surgical systems. AMBF is a relatively newer research tool and applying novel robots within the environment requires extensive debugging. The model for the Galen robot was pulled into AMBF, and the robot files were updated to be compatible with our controller and simulation. A pipeline and instructions were developed to aid in the use of MATLAB with AMBF for controlling the Galen robot using ROS nodes. For these instructions see the following files:

TOGAC25_AMBF_IN_AMBFforGalen

TOGAC29_AMBF_IN_ConnectMatlab2AMBF

TOGAC50_AMBF_in_AdmittanceControlTeleop

With an interface for interacting with the Galen robot created testing of simulated admittance controllers in real time could be performed. Using our developed galen_interface() methods we controlled the Galen robot in AMBF with MATLAB over ROS at ~200Hz. Our intent was to control the system first with joint control, then develop our own Cartesian control package. We started in one dimension to evaluate the system. Initially, we chose the up and down Z dimension and turned off simulated gravity to mimic the gravity compensation found in the real controller. With AMBF acting as the lower or mid level controller, MATLAB served as the high level controller for admittance. We identified the system of the simulated Galen robot in the z direction with both a chirp and step response to ensure proper characterization of the linear transfer function. The stability of the identified function was analyzed as well as the transparency. We compared our results with the stability Analysis of [2], however we were unable to properly reproduce the results of [2] and the conclusions of stability maps feasibility is somewhat uncertain because of this. The methods of [2] do not adequately describe their stability analysis and make it difficult to understand how they intend to account for multi DOF control stability. The summary of this analysis and the results can be found in TOGAC36_AMBF_R_AydinStab.

With the 1 DOF system identified in AMBF, we were able to create a stability and cost map in joint space to determine acceptable m_ad and b_ad, the desired admittance values. We verified that anticipated stable admittance values were stable during simulated worst case human and environment impedance, and we saw instability with the predicted unstable admittance values during simulated impedance. These results gave us the confidence in the feasibility of this impedance result, and the work towards identifying the whole system, transitioning to Cartesian control, and applying the controller to the real robot was started. The results of this can be seen here in the following report. TOGAC34_AMBF_R_LumpedSystemIdentification

To implement velocity Cartesian control in simulation, the Jacobian of the simulated robot was needed. The forward kinematics were found for the Galen robot in simulation using the prismatic robot case found in [12] and the jacobian was found by transforming from the result of [12] to the end effector of the Galen robot. With the jacobian in place for control of the 5 joints, the full system was identified with chirp velocity inputs applied individually to the 5 DOF. The systems were identified using MATLAB's system identification toolbox just as in the previous cases. The stability maps for all 5 DOF were generated. Admittance gains were selected from each figure and tested in impedance situations with galen robot controller in simulation. The results for these tests, figures, and the derivation of the forward kinematics can all be found in the following files.

TOGAC51_AMBF_R_CartesianSystemIdentification.

TOGAC40_D_DeltaKin

Additionally, functionality was added to the admittance controller design to include the use of teleoperation via a GUI and joystick. For instructions on how to use this see the following.

TOGAC50_AMBF_in_AdmittanceControlTeleop

Single Axis Galen Implementation of Admittance Control

Time allotted for the implementation and testing of one single axis of the Galen robot. The Z dimension was chosen and used to evaluate the admittance control’s ability to interpret the user’s input into a velocity reference.

The following transfer functions for the X, Y, and Z axis were obtained for the real Galen robot system identification. Note that the actual system features a few new errors introducing variables. Two prominent variables that will be addressed here are the delay and the motor backlash.

The tfest MATLAB estimator was able to estimate the transfer functions in addition to measuring the time delays. This time delay could be caused by several factors. A few of these factors are due to the control processing delay between communication from the mid-level control to low-level to ROS and the actual robot. The delay is also a result of the lagging time in the acceleration or inertia of the robot hardware. This is especially present in the case of motor backlash. When the commanded velocity or present velocity of the robot reaches zero, especially on robot initialization, commands that move the robot out of a stationary state must first overcome the effects of static friction. This is called motor backlash and can cause significant delays and shifting of the output signal.

The output position of the real Galen robot, the velocity, and an fft of the velocity noise

Observe the above figure for the velocity input and output plot. The motor backlash delay can be seen at any point where the velocity out reaches zero. Until the robot can break the static friction and overcome the initial motor inertia, the velocity controller must quickly compensate to catch up to the reference.

We also can see that the output velocity of the robot, in orange, is a rather noisy signal. This is due to the fact that we only have access to the position measuring crtk_command tools that observe the motor encoders. To get the velocity of the position signal, we use a finite differentiation method between the current timestep and the previous time step to derive the position.This introduces lots of noise, and in order to utilize a more precise velocity signal for feedback control it is important to filter the data.

We ran an fft on the finite differentiated velocity to observe what appropriate cutoff frequency should be used and deemed 75 Hz was an appropriate frequency to use in a first order low pass filter.

The above figure also shows an example of the response of the Z axis identification. This axis showed the smallest time delay of the 3 identified systems. We notice that this time delay is large rather large on the X axis and there are many factors still to be investigated that could be causing this. There is much more work to be done on the real system implementation and testing.

For our current approach, we chose to start simply with a single axis where we were most confident to obtain a good admittance response. We found the Z to have the smallest time delay and the stability analysis was deemed the most stable according to the following maps.

Upper bound (with human and environment) impedance and Lower bound (only human) impedance stability and transparency maps for the Galen z axis.

With the above results we were able to select a set of gains for the human only contact impedance case and test them on the real Galen robot.

User interacting with the Galen robot end effector to apply a wrench

The admittance gains of 8kg and 20Ns/m were chosen for the mass and damping values respectively. A test was run for 10 seconds with the robot in gravity compensation mode. The user was instructed to wait 5 seconds and then apply an upwards input on the robot end effector. The following plot was obtained for the input reference velocity vs the input user force.

The recorded input reference velocity as calculated from the admittance controller and the measured true force from the Galen robot end effector sensor.

As anticipated, the robot tracked the force references and converted the input wrench to a clean and smooth velocity trajectory. Notice that even with the Galen gravity compensation on, during the 0 to 4 second period without human contact, the robot experiences a small downward force. This indicates that the Galen robot needs to be recalibrated or adjusted to better handle gravity compensation.

The present gains for the admittance controller in this Z axis show a good level of admittance that feels smooth, light, and controllable. Qualitatively, compared to Z motions on the current on-board Galen controller, our controller is noticeably better at feeling more transparent. However true testing and evaluation of the controllers will prove if the results truly are more transparent.

All of the minimum deliverables in the project were completed and delivered in the expected time frame.

The expected deliverables are in progress. System Identification has been done for the X, Y, and Z DOF of the real system, and a prototype controller is on board the Galen Robot that has yet to be thoroughly tested. No work on the virtual fixtures has been attempted yet since the controller is not running. The stability analysis of the identified systems has been done, but they are not yet verified.

The maximum deliverables have not been achieved during the duration of this course.

Due to the time spent developing the controller theory, testing in MATLAB and AMBF, and the caution surrounding the real hardware, the maximum and some expected results were not achieved. Priority was given to ensure that the proof of concept behind the admittance control design was truly feasible. We have been able to show this much as outlined in this report and the others found on the project wiki.

Other key factors outside the scope of our project became important subtasks for each deliverable. These greatly extended the time of development as well. For instance, we were unaware of the rigorous testing and analysis we would need to perform system identification in both simulation and on the real hardware. The execution of this task took longer than expected and was not originally in scope. In addition to the system identification, the maximum deliverable used to contain a user study that was in deliberation for some of the early weeks of the project. The decision to drop the user for the time being freed up time to focus on the development tasks, but we expect to reconsider adding the user study in our maximum deliverables again once the real Galen Robot admittance controller is functional.

With the project in its current state, the software, development, design, and instructions are all well documented and available for review and reference. Future work will be done to complete the outstanding deliverables post spring 2023 semester until the beginning of 2023 summer.

Discussion

Overall, the project is a success. We were able to accomplish a significant amount of work with only a few hands. I had the help of great mentors who helped streamline the project and direct me to the resources I needed. We succeeded in generating a working Admittance controller in the AMBF simulation that gives us the expected performance we would like with the real hardware. In addition to the controller in simulation, tools to connect the robot to AMBF and teleoperate the system with easily repeatable tools will now allow us to perform many different control tests faster and repeatably.

The control framework presented in the AMBF simulation is applicable to more robots than the Galen Robot. This MATLAB interface can be extended in use for any serial robot or other AMBF robots. The controller dynamics could easily be updated, the stability and transparency maps can similarly be calculated with these tools, and the admittance gains can simply be selected. This will make applying admittance controllers to any form of cobot relativley simple.

While the controller has been verified in contact in the simulation, it still would be nice to have seen this fully implemented on the real hardware. The efforts to do so are still on going, and the results have yet to be seen. We do anticipate the implementation of the controller to be promising as we have observed in the Z axis control on the Galen robot. With the full onboard admittance controller repeatable testing of the real hardware with the old controller and our new controller will allow us to capture results that show a measurable improvement.

Next Steps

Several next steps have been discussed throughout this report and will be summarized here again.

A few more steps are necessary to ensure that the AMBF simulation controls well for the robot. It has been shown to work in simulated contact, but it would be very interesting to study the performance of the robot in the AMBF simulation with different forms of contact. If AMBF features tools to control the stiffness, penetration depth, and angle of contact, we could further investigate how the robot performs in contact, that may better predict how it would respond to contact in the real world. This contact could also be done in more rigorous ways by moving the robot in several DOF at different rates and impedance values.

If we are able to more accurately see how the robot interacts with a simulated environment, there are ways we could use the simulated objects in AMBF to create virtual fixtures. We could ‘twin’ the two control systems, to possibly use this controller as a virtual fixture. With objects in contact in AMBF, we could run the real Galen Robot in parallel and observe when they both reach certain positions and velocities. If there is contact in AMBF, we could impose a virtual contact on the real system that would stop the robot and push it back.

This, of course, requires us to first have the controller also working on the actual Galen robot. The next steps in scope of the project completion before summer 2023 are to have the real Galen robot controller running and performing similarly to the AMBF simulation. In turn we should be able to select admittance gains that improve the transparency and stability of the Galen robot.

With the Galen robot controller verified and the other aspects of the simulation environment running as we would like, we will consider performing a user study to asses the qualitative response of the robot. We will have ENT surgeons operate the robot with the previously implemented Galen robot controller and also operate it with our new admittance controller. We will design a series of tests consisting of repeatable and measurable tasks where we will measure their performance. We will also ask for their feedback on the system.

In conclusion, there is still much work to be done to get the controller working on the real robot. This work will be continued. All the design documents, reports, and software are present on the course website and available for my mentors through the LCSR. Any code written specifically for control of the Galen Robot is on the computer in the MOCK OR. I will continue to work on the project until the beginning of summer as well to complete the controller implementation and publish the work we have done in admittance control development.

• Project Plan
  • Project plan presentation
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
  • Paper for review: Aydin et al.
  • Paper review report
• Project Checkpoint
  • Project checkpoint presentation
  • Project checkpoint presentation
• Paper Seminar Presentations
  • Paper review presentation
  • Paper review presentation
• Poster teaser
  • Poster Teaser presentation
• Project Final Presentation
  • PDF of Poster
• Project Final Report
  • Final Report
• Project Mentor's Report
  • Mentor's Report

• [1] Alonso, V., & de la Puente, P. (2018). System transparency in shared autonomy: A mini review. Frontiers in Neurorobotics, 12. https://doi.org/10.3389/fnbot.2018.00083
• [2] Aydin, Y., Sirintuna, D., & Basdogan, C. (2020). Towards collaborative drilling with a Cobot using admittance controller. Transactions of the Institute of Measurement and Control, 43(8), 1760–1773. https://doi.org/10.1177/0142331220934643
• [3] Feng, A. L., Razavi, C. R., Lakshminarayanan, P., Ashai, Z., Olds, K., Balicki, M., Gooi, Z., Day, A. T., Taylor, R. H., & Richmon, J. D. (2017). The Robotic Ent Microsurgery System: A novel robotic platform for Microvascular Surgery. The Laryngoscope, 127(11), 2495–2500. https://doi.org/10.1002/lary.26667
• [4] Huang, S. H. (2019). Optimizing for Robot Transparency. UC Berkeley. ProQuest ID: Huang_berkeley_0028E_19188. Merritt ID: ark:/13030/m5mq08z2. Retrieved from https://escholarship.org/uc/item/5q20h9cs
• [5] J. Fong, V. Crocher, Y. Tan, D. Oetomo and I. Mareels, “EMU: A transparent 3D robotic manipulandum for upper-limb rehabilitation,” 2017 International Conference on Rehabilitation Robotics (ICORR), London, UK, 2017, pp. 771-776, doi: 10.1109/ICORR.2017.8009341.
• [6] Keemink AQ, van der Kooij H, Stienen AH. Admittance control for physical human–robot interaction. The International Journal of Robotics Research. 2018;37(11):1421-1444. doi:10.1177/0278364918768950
• [7] Democratizing Robotic Surgery and Microsurgery. Galen Robotics. (2022, November 15). Retrieved February 13, 2023, from https://www.galenrobotics.com/
• [8] Simple gantt chart. Microsoft 365 Templates. (2022, August 13). Retrieved February 13, 2023, from https://templates.office.com/en-us/simple-gantt-chart-tm16400962
• [9] Munawar, A., Wang, Y., Gondokaryono, R., & Fischer, G. S. (2019). A real-time dynamic simulator and an associated front-end representation format for simulating complex robots and environments. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). https://doi.org/10.1109/iros40897.2019.8968568
• [10] Munawar, A., Wu, J. Y., Fischer, G. S., Taylor, R. H., & Kazanzides, P. (2022). Open simulation environment for learning and practice of robot-assisted surgical suturing. IEEE Robotics and Automation Letters, 7(2), 3843–3850. https://doi.org/10.1109/lra.2022.3146900
• [11] Varier, V. M., Rajamani, D. K., Tavakkolmoghaddam, F., Munawar, A., & Fischer, G. S. (2022). AMBF-RL: A real-time simulation based Reinforcement Learning Toolkit for Medical Robotics. 2022 International Symposium on Medical Robotics (ISMR). https://doi.org/10.1109/ismr48347.2022.9807609
• [12] Williams, R. L. (2016). The Delta Parallel Robot: Kinematics Solutions. Mechanical Engineering: Ohio University, 1–46.

Here is a list of other project files (e.g., source code, documentation) associated with the project.

If the file your searching for is not listed here, check the github repo: https://gitfront.io/r/user-6241796/DtaCrgWCApHi/CIS2-Admittance-Control/

togac1_d_deliverablestable.xlsx

togac2_d_documentcontrol.xlsx

togac3_d_arvidsteininpaper.pdf

togac4_d_aydinetalpaper.pdf

togac6_cd_controlsystemsdesign.pdf

togac5_d_groupandtaskdependencylist.pdf

togac7_d_galentransparencycontrolganttchart.xlsx

togac8_p_projectplanpresentation.pptx

togac9_d_projectphysicaldependencies.xlsx

togac10_d_projectproposal.pdf

togac11_msd_r_admitconfeedbackandforwardvsfeedback.pdf

togac17_msd_r_stabilityfbandffapproach.pdf

togac22_r_paperreviewreport.pdf

togac23_p_paperreviewpresentation.pptx

togac25_ambf_in_ambfforgalen.pdf

togac27_cd_transferfunctionderivandstability.pdf

togac29_ambf_in_connectmatlab2ambf.pdf

togac34_ambf_r_lumpedsystemidentification.pdf

togac36_ambf_r_aydinstab.pdf

togac40_d_deltakin.pdf

togac50_ambf_in_admittancecontrolteleop.pdf

togac51_ambf_r_cartesiansystemidentification.pdf